[Feature Request]: Add otelAgentGateway.service.annotations support in datadog Helm chart

[carlcauchi]

Pre-submission Checklist

• I have searched existing issues and this is not a duplicate
• This is a Helm chart issue, not a Datadog product/service problem

Helm chart name

datadog

Helm chart version

3.179.0

Bug Report

Summary

When enabling the OTel Agent Gateway (otelAgentGateway.enabled=true) in the datadog/datadog Helm chart, there is currently no way to configure Service annotations for the generated *-otel-agent-gateway Service.

This makes it difficult to properly configure cloud load balancers (e.g. AWS NLB, Azure LB, GCP LB), which rely on Service annotations.

Steps to Reproduce

Current Behavior

The otel-agent-gateway Service is created in:

charts/datadog/templates/agent-services.yaml

The template renders:

apiVersion: v1
kind: Service
metadata:
  name: {{ template "datadog.fullname" . }}-otel-agent-gateway
spec:
  type: {{ .Values.otelAgentGateway.service.type }}

The chart allows configuring:

otelAgentGateway:
  service:
    type: ClusterIP | NodePort | LoadBalancer

However, there is no support for metadata.annotations, which prevents configuration of cloud-provider-specific load balancer behavior.

Why This Is Needed

In AWS EKS (using AWS Load Balancer Controller), NLB configuration requires Service annotations such as:

service.beta.kubernetes.io/aws-load-balancer-type: "external"
service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: "ip"
service.beta.kubernetes.io/aws-load-balancer-scheme: "internal"

Without annotation support, users must:

Patch the rendered Service (Helm post-render / ArgoCD patch), OR

Create a separate Service targeting the same pods

Both approaches are workarounds and add operational complexity.

Expected Behavior

The chart should allow:

otelAgentGateway:
  enabled: true
  service:
    type: LoadBalancer
    annotations:
      service.beta.kubernetes.io/aws-load-balancer-type: "external"
      service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: "ip"
      service.beta.kubernetes.io/aws-load-balancer-scheme: "internal"

Which would render:

metadata:
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-type: external
    service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: ip
    service.beta.kubernetes.io/aws-load-balancer-scheme: internal

Proposed Implementation

Add annotation support similar to other services in the chart:

metadata:
  name: {{ template "datadog.fullname" . }}-otel-agent-gateway
  {{- with .Values.otelAgentGateway.service.annotations }}
  annotations:
{{ toYaml . | nindent 4 }}
  {{- end }}

And extend values.yaml:

otelAgentGateway:
  service:
    type: ClusterIP
    annotations: {}

Environment

Environment

Chart: datadog/datadog

Kubernetes: EKS

Load Balancer: AWS NLB (AWS Load Balancer Controller)

Use case: Exposing OTLP (gRPC/HTTP) externally or internally via NLB

Additional Context

Additional Context

Other Services in Kubernetes commonly expose service.annotations in Helm charts to allow provider-specific LB configuration. Adding this would improve flexibility and remove the need for post-render patching.