Agent

Prelude

Release on: 2025-09-10

• Please refer to the 7.70.1 tag on integrations-core for the list of changes on the Core Checks

Bug Fixes

• gpum: fix patching of SystemD device permissions

Datadog Cluster Agent

Prelude

Released on: 2025-09-10 Pinned to datadog-agent v7.70.1: CHANGELOG.

Agent

Prelude

Release on: 2025-09-03

• Please refer to the 7.70.0 tag on integrations-core for the list of changes on the Core Checks

Upgrade Notes

• GPUM: GPU workload metadata and autodiscovery are now enabled by default. If you want to disable this functionality, add "nvml" to the autoconfig_exclude_features configuration flag in your datadog.yaml file.
• Upgraded the WDK version used by ddnpm.
• Upgrade ddprocmon to version 1.3.0. This version contains minor cosmetic changes to the ABI.
• None. This change is backward-compatible and does not modify behavior unless users explicitly set new environment variables.

New Features

• Windows: Add certificate chain validation to Windows Certificate Store integration.
• Added agent remote-config reset command to reset the remote configuration state which can be useful in troubleshooting issues related to remote configuration.
• GPUM: Add new configuration flag gpu.enabled to agent config to control the GPU Core-Check component, disabled by default.
• Adds Cisco IPsec VPN tunnel metrics and options data collection to SNMP. This can be enabled and disabled using the collect_vpn config.
• Support Cloud Run Jobs in serverless-init.
• The Windows Agent install script (Install-Datadog.ps1) now supports setting host tags and enabling logs collection at install time via the DD_TAGS and DD_LOGS_ENABLED environment variables. This allows users to configure tags and logs without post-install edits to datadog.yaml.

Enhancement Notes

• Add cluster_arn, service_arn, and task_arn tags to the ECS tasks.
• Add container_instance_arn as a host tag when running on ECS EC2.
• Add a script to manage IIS .NET APM instrumentation on Windows
• Add macOS build for AArch64/ARM64.
• Added garbage collection of the tmp directory to the datadog-installer.
• The Datadog autoscaler now can apply vertical scaling recommendations to sidecar init containers.
• Agents are now built with Go 1.24.6.
• Expose additional cgroup memory metrics: active_anon, inactive_anon, active_file, inactive_file, unevictable, shmem, file_mapped, file_dirty, file_writeback, page_tables, refault_anon, refault_file. Metrics are disbaled be default. To enable them please modify conf.d/container.d/conf.yaml and flip extended_memory_metrics to true.
• Add enhanced metrics for task started, ended, and duration for Cloud Run Jobs.
• gpu: Renamed gpu.core.usage and gpu.memory.usage to gpu.process.core/memory.usage
• If a path is specified in the URL in logs_config.logs_dd_url, use it as a prefix.
• Add the datadog.snmp.requests metric, which tracks the number of SNMP requests sent by the Agent to devices.
• Added detection and replacement of tags: blocks in datadog.yaml to ensure only top-level, uncommented, and non-nested tags are modified during install.
• Add thumbprints and serial numbers as tags for certificates and CRLs to the Windows certificate store integration.
• Remote Agent management on Windows now automatically retries when the MSI returns error 1601 (ERROR_INSTALL_SERVICE_FAILURE).
• Remote Agent Management on Windows now automatically retries when an MSI custom action returns error 1601 (ERROR_INSTALL_SERVICE_FAILURE) and causes Error 1719. The Windows Installer Service could not be accessed.

Deprecation Notes

• Remove deprecated serializer options.
  • enable_stream_payload_serialization
  • enable_service_checks_stream_payload_serialization
  • enable_events_stream_payload_serialization
  • enable_sketch_stream_payload_serialization

Bug Fixes

• Fix a panic in Docker file log parsing when received messages are null equivalent.
• When the Autodiscovery scheduler gets blocked, it triggers a health check failure (reported in the agent health command).
• When the prometheus_scrape.enabled option is set to true and the pod has a prometheus.io/port annotation, containers must expose that port in their spec in order for the Agent to schedule openmetrics checks.
• Fixes an issue where pending ECS tasks would temporarily remove the global tags ecs_cluster_name and cluster_arn.
• Fix bug causing integrations configured via docker labels or K8s annotations to be overwritten in the agent so logs from integrations now work when configured using via docker labels or K8s annotations.
• Fix noisy error logs when scrubbing large pod container commands.
• Remote updates on Windows now quotes and escapes MSI property values. This fixes an issue that caused remote updates to fail when the Agent username contains whitespace, for example DDAGENTUSER_NAME=NT AUTHORITY\SYSTEM.
• Remote Agent updates on Windows now handle computer name updates done after the initial install of the Agent. This allows the use of golden images to deploy remotely updatable hosts.
• gpu: Java programs using deeplearning4j are now properly detected and monitored
• gpu: fix container-to-device assignment for Docker containers
• Data Streams Messages: Fix Unscheduling of kafka_consumer integrations that was leading to duplicate integrations running in parallel.
• The Kafka messages feature is not working in some cases when using K8s labels because config is translated to yaml.
• Load check tags onto the orchestrator check configured to be dispatched and run as a cluster check
• APM: Fix issue where the probabilistic sampler would drop all OTLP traffic.
• Kubernetes logs source timestamps are now validated. Invalid timestamps will not be passed.
• The Agent no longer logs an error each time it fails to connect to system-probe through \\.\pipe\dd_system_probe. This log message was added in Agent 7.63.0. The error is still logged if it persists long enough, see [#35790](#35790). Each failed connection attempt still logs the error at debug level to aid troubleshooting.
• Fixed an issue where DD_TAGS and DD_LOGS_ENABLED were silently ignored during install, requiring manual configuration edits.
• Fixed an issue where metrics in the Windows certificate store integration are merged when multiple certificates have the same subject.
• Fixed an issue where the Windows Event Log tailer could lose historical events (the 'amnesia bug') when it started with no prior bookmark and no events were processed immediately. The Agent now correctly seeds an initial bookmark from the most recent event upon startup, ensuring no events are missed on subsequent runs.

Other Notes

• Add handling for dbm-health events in the event platform forwarder. These events are used by Database Monitoring integrations to report on Agent status.
• Add a new metric counter to the Agent telemetry for transactions.errors. This will help to monitor the rate of empty or rejected sketch payloads sent to the backend.

Datadog Cluster Agent

Prelude

Released on: 2025-09-03 Pinned to datadog-agent v7.70.0: CHANGELOG.

Enhancement Notes

• Created a new configuration option orchestrator_explorer.custom_resources.max_count that will allow customers to increase the number of custom resources they can collect in the Cluster Agent.
• Time zones are now collected for CronJobs in the orchestrator check
• Collect EndpointSlices by default.

Bug Fixes

• Correctly set the VerticalPodAutoscaler mode to "Auto" when not provided
• Fixed an issue that caused some Datadog Cluster Agent replicas to restart during the first install.
• When using leases for leader election with the leader_election_default_resource option, the Datadog Cluster Agent no longer creates an unused config map.
• Fix the cronjob.on_schedule_check service check of the kubernetes_state check for CronJobs that have a specific time zone.

Agent

Prelude

Release on: 2025-08-27

• Please refer to the 7.69.4 tag on integrations-core for the list of changes on the Core Checks

Upgrade Notes

• Upgrade sqllite to 3.50.4 on Windows.

Datadog Cluster Agent

Prelude

Released on: 2025-08-27 Pinned to datadog-agent v7.69.4: CHANGELOG.

Agent

Prelude

Release on: 2025-08-26

• Please refer to the 7.69.3 tag on integrations-core for the list of changes on the Core Checks

Enhancement Notes

• Agents are now built with Go 1.24.6.

Security Notes

• JMXFetch upgrade to 0.50.0 to mitigate against CVE-2025-52999.

Bug Fixes

• Fixes a bug in the Trace Agent that prevented DDOT from shutting down gracefully.
• Fixed truncation telemetry so it records service and source values correctly, by moving metrics recording to the tailer layer.

Datadog Cluster Agent

Prelude

Released on: 2025-08-26 Pinned to datadog-agent v7.69.3: CHANGELOG.

Agent

Prelude

Release on: 2025-08-20

• Please refer to the 7.69.2 tag on integrations-core for the list of changes on the Core Checks

Upgrade Notes

• Upgrade libxml2 to 2.14.5.

Datadog Cluster Agent

Prelude

Released on: 2025-08-20 Pinned to datadog-agent v7.69.2: CHANGELOG.

Agent

Prelude

Release on: 2025-08-18

• Please refer to the 7.69.1 tag on integrations-core for the list of changes on the Core Checks

Bug Fixes

• Remote Agent updates on Windows now checks if the custom Agent password is available before updating the Agent. This prevents the update from failing later and leaving the host without an Agent installed.

  This is only applicable to domain-joined hosts using a custom Agent username and password. See [Agent 7.66.0 upgrade notes](https://github.com/DataDog/datadog-agent/releases/tag/7.66.0) for more information.

Datadog Cluster Agent

Prelude

Released on: 2025-08-18 Pinned to datadog-agent v7.69.1: CHANGELOG.

Agent

Prelude

Release on: 2025-08-14

• Please refer to the 7.69.0 tag on integrations-core for the list of changes on the Core Checks

Upgrade Notes

• The cilium conntracker is now enabled by default in the system-probe, and now expects the /sys/fs/bpf to be mounted at /host/sys/fs/bpf in containerized environments. The conntracker, if enabled, will fail to load unless this mount is provided, with the log line "not loading cilium conntracker since cilium maps are not present" in system-probe's log file. Users who have enabled this feature can either upgrade to the latest helm chart or add this mount to their container

New Features

• Adds additional information and data related to the setsockopt hook.
  • Socket Information:
    • Socket type
    • Socket family
    • Socket protocol
  • Filter Information
    • Disassembled filter
    • Filter hash
• You can now set the JAVA_TOOL_OPTIONS that JMXFetch uses by setting the jmx_java_tool_options configuration option in the datadog.yaml config file. This allows you to pass additional JVM options to JMXFetch, such as memory settings or system properties.
• Adding a TracerPayloadModifier to the Trace Agent.
• pkg/trace/api: Container tags hash is returned as a response header of the info endpoint.
• Added new config option include_ephemeral_containers to collect Kubernetes ephemeral containers. The option is disabled by default. When enabled, the Agent will report container.* and kubernetes.* metrics for ephemeral containers. It will also collect logs and schedule checks for ephemeral containers when configured to do so.
• Data Streams Monitoring: Adds new feature allowing users to retrieve messages from Kafka topics.
• Change collect_gpu_tags config flag to be enabled by default. Now the Agent collects an additional gpu_host host tag for all hosts that have Nvidia GPUs.
• Added new processing rule to omit truncated logs from being sent to ingest
• GPU: Add GPM collector for Hopper and newer NVIDIA GPUs
• Adds VPN tunnels and route table data collection to SNMP. This can be enabled/disabled using the collect_vpn config.
• The NTP check on Windows now discovers the primary domain controller (PDC) on domain-joined hosts when use_local_defined_servers is enabled. If the PDC is unavailable, it automatically falls back to registry-defined servers. Check now performs order-insensitive server list comparisons, reduces log noise, and avoids using itself as a time source when running on a domain controller.
• [Preview] The agent can now connect to the AWS SSM, AWS Secrets, Hashicorp Vault and Azure Keyvault secret management solutions to resolve secrets without requiring a user provided binary. For this, two new settings are introduced: secret_backend_type and secret_backend_config. For more information see: https://docs.datadoghq.com/agent/configuration/secrets-management
• Added support in DDOT for the datadogexporter.proxy_url configuration option. This allows users to specify proxy settings for DDOT with the collector configuration.
• Windows: Add CRL monitoring to the Windows Certificate Store integration.

Enhancement Notes

• The serverless-init build uses the new TracerPayloadModifier to add Function Tags to the _dd.tags.function tag of the Tracer Payload to support serverless trace tagging.
• Agents are now built with Go 1.24.5.
• The user is now able to specify which features they want enabled inside of the converter. Previously, the user would have to either enable or disable everything.
• DDOT now uses zstd compression for logs by default.
• If a check has both a Go and a Python version, the Go version now has priority by default. This change should not have any visible impact, but if needed, you can disable this configuration by setting prioritize_go_check_loader to false.
• GPUM: the "status" command now returns status of the system-probe part of GPU monitoring
• Added new DogStatsD configuration option "dogstatsd_flush_incomplete_buckets". When enabled, DogStatsD will flush all received metrics during shutdown, regardless of which time-interval based bucket they belong to.
• Agent integration metadata payloads now include the JMX integrations.
• Allow users to configure the HTTP timeout for the Logs Agent.
• No longer have the Logs Agent fall back to TCP when configuring logs_config.logs_dd_url with a http(s):// prefix.
• If the Oracle can_connect check is critical, also set the can_query check to critical.
• Display the number of times each log processor has been used in the Logs Agent status endpoint.
• Reduce binary size by removing the Sensitive Data Scanner (SDS) from the logs agent.
• OTLP spans support db.namespace semantic and map to db.name for DBM support.
• Generate a more detailed warning when the Logs Agent tailer limit is reached.
• Improved the granularity of the Logs Agent pipeline monitor to track the capacity of each individual component of the pipeline.
• Remote Agent management operations on Windows now attempt to force stop the Agent services if they do not respond to Service Control Manager requests.
• Remote Agent management on Windows now automatically retries when the MSI returns error 1618 (ERROR_INSTALL_ALREADY_RUNNING).

Bug Fixes

• Correctly respect the ecs_collect_resource_tags_ec2 variable when calling the ECS Agent. Start caching tags to reduce burden on the ECS Agent. Start logging error responses from the ECS agent.
• Fix a panic in Docker streams log parsing when stream messages are truncated on transmission.
• Fix the cgroup reader bug that would prevent the generic container check from sending metrics when the Agent encountered a permission error.
• Fixes invalid logs compression error in DDOT, sets DDOT logs compression to gzip.
• Add support for selecting the endpoint resolution method using advanced AD identifiers in Kubernetes endpoint check configurations defined in files or configmaps. This enables static pod check configurations to correctly resolve the endpoint by setting resolve method to "ip".
• Fixed the serializer exporter for the OSS Collector, which was not setting the correct proxy variables when sending metric data.
• Fixed Windows installer overwriting install_info from setup scripts. When using Fleet Automation setup scripts, the subsequent MSI installation now skips writing install_info via a new SKIP_INSTALL_INFO flag, preserving the original setup script installation method tracking.
• Fix Jetson check to correctly parse the output of tegrastats for Orin boards.
• Fix incorrect container.memory.kernel value when running with Kernel >= 5.19 and cgroupv2
• Breaking change - Fixes the Oracle service name tag to be service_name instead of service. This corrects the conflict with the APM service tag. This is a breaking change for any users who had been relying on the service tag to be set to the Oracle service name. The service tag can still be set explicitly in the tags configuration if needed.
• Metrics sent from the process check on the core agent now have the host tag.
• GPU: fix a bug where the device assigned to a process could be wrong if it updates the CUDA_VISIBLE_DEVICES environment variable during runtime
• GPUM: fix Kubernetes device allocation detection in Google Kubernetes Engine
• The NTP check will no longer fail to start if the initial discovery of local NTP servers fails at agent startup.
• Limit the HTTP timeout on startup to 5 seconds for the Logs Agent.
• Prevent the process component from running in the cluster worker.
• Removes an extra copy of agent.exe from the Windows container
• Remote Agent management operations on Windows now attempt to restart the Agent services after failing to stop the services or uninstall the Agent.
• Fix Cgroup namespace not properly detected in Workload Protection, leading to incorrect container ID resolution and misqualified detections.

Other Notes

• Add a new metric to the Agent telemetry for the startup and running states. This will help us track the startup and running states of the Agent.
• Transparent Huge Pages (THP) usage is now disabled by default in the System Probe and Security Agent. To re-enable their usage, set the system_probe_config.disable_thp or security_agent.disable_thp configuration options to false.

Datadog Cluster Agent

Prelude

Released on: 2025-08-14 Pinned to datadog-agent v7.69.0: CHANGELOG.

Enhancement Notes

• The auto-instrumentation webhook supports labels and annotations as tags configuration. If any of the label or annotation mappings for the incoming pod correspond to Universal Service Tags (service, env, or version), the webhook will also add the corresponding UST environment variable to the pod (DD_SERVICE, DD_ENV, or `DD...

Agent

Prelude

Release on: 2025-07-28

• Please refer to the 7.68.3 tag on integrations-core for the list of changes on the Core Checks

Upgrade Notes

• Upgraded JMXFetch to 0.49.9 <https://github.com/DataDog/jmxfetch/releases/0.49.9> which reverts a change that broke support for Java 7. See 0.49.9 <https://github.com/DataDog/jmxfetch/releases/tag/0.49.9> for more details.

Enhancement Notes

• Agents are now built with Go 1.24.5.

Bug Fixes

• The Agent MSI no longer fails when it is unable to read the version information from [MsiGetProductInfo](https://learn.microsoft.com/en-us/windows/win32/api/msi/nf-msi-msigetproductinfow).

  When upgrading from an Agent version earlier than 7.56, a failure in MsiGetProductInfo, combined with an MSI rollback, may result in missing system drivers. If this occurs, repairing or reinstalling the Agent will restore the missing drivers. This issue is typically caused by a corrupted Windows Installer registry. To repair these issues, refer to the [Microsoft Program Install and Uninstall Troubleshooter](https://support.microsoft.com/en-us/topic/fix-problems-that-block-programs-from-being-installed-or-removed-cca7d1b6-65a9-3d98-426b-e9f927e1eb4d).

Datadog Cluster Agent

Prelude

Released on: 2025-07-28 Pinned to datadog-agent v7.68.3: CHANGELOG.

Agent

Prelude

Release on: 2025-07-21

• Please refer to the 7.68.2 tag on integrations-core for the list of changes on the Core Checks

Bug Fixes

• Fix an issue with the Agent pre-install script that caused integrations shipped with the Agent to be removed during an Agent upgrade.
• Print the correct FIPS status for the Cluster Agent when running in FIPS mode.

Datadog Cluster Agent

Prelude

Released on: 2025-07-21 Pinned to datadog-agent v7.68.2: CHANGELOG.

Agent

Prelude

Release on: 2025-07-17

• Please refer to the 7.68.1 tag on integrations-core for the list of changes on the Core Checks

Security Notes

• Bump the secret-generic-connector side binary to 0.2.5

Datadog Cluster Agent

Prelude

Released on: 2025-07-17 Pinned to datadog-agent v7.68.1: CHANGELOG.