Closed
Description
Component(s)
pkg/datadog
What happened?
Description
Kubernetes restarts pod after 30 seconds when deployed as daemonset with Datadog Exporter enabled, if exporters::datadog::hostname is not set
Steps to Reproduce
- edit examples/kubernetes/daemonset-collector-dev.yaml to include a datadog exporter with an API key
make kind-buildmake kind-install-daemonset
Expected Result
pod starts up with some sort of system hostname detected
Actual Result
pod restarts every 30 seconds
Collector version
v0.127.0 (main)
Environment information
Environment
OS: Linux/Kubernetes daemonset
OpenTelemetry Collector configuration
mode: daemonset
image:
repository: otelcontribcol-dev
tag: "0.0.1"
pullPolicy: IfNotPresent
command:
name: otelcontribcol
extraEnvs:
- name: K8S_NODE_NAME
valueFrom:
fieldRef:
fieldPath: spec.nodeName
- name: DD_SITE
value: "datad0g.com"
- name: DD_API_KEY
valueFrom:
secretKeyRef:
name: datadog-secret
key: api-key
clusterRole:
create: true
rules:
- apiGroups:
- ''
resources:
- 'pods'
- 'nodes'
verbs:
- 'get'
- 'list'
- 'watch'
- apiGroups: [ "" ]
resources: [ "nodes/stats", "nodes/proxy"]
verbs: [ "get" ]
config:
exporters:
debug:
verbosity: detailed
datadog:
api:
site: ${env:DD_SITE}
key: ${env:DD_API_KEY}
receivers:
kubeletstats:
collection_interval: 10s
auth_type: 'serviceAccount'
endpoint: '${env:K8S_NODE_NAME}:10250'
insecure_skip_verify: true
k8s_api_config:
auth_type: serviceAccount
metrics:
container.cpu.utilization:
enabled: true
container.cpu.time:
enabled: true
container.cpu.usage:
enabled: true
k8s.container.cpu_limit_utilization:
enabled: true
service:
pipelines:
metrics:
receivers: [kubeletstats]
processors: [batch]
exporters: [debug, datadog]Log output
john.peterson@COMP-HHVKJYJXCY opentelemetry-collector-contrib % kubectl logs daemonset-collector-dev-opentelemetry-collector-agent-zs768
2025-05-29T17:54:47.545Z info [email protected]/service.go:199 Setting up own telemetry... {"resource": {}}
2025-05-29T17:54:47.546Z info builders/builders.go:26 Development component. May change in the future. {"resource": {}, "otelcol.component.id": "debug", "otelcol.component.kind": "exporter", "otelcol.signal": "traces"}
2025-05-29T17:54:47.546Z info [email protected]/memorylimiter.go:149 Using percentage memory limiter {"resource": {}, "otelcol.component.kind": "processor", "total_memory_mib": 15972, "limit_percentage": 80, "spike_limit_percentage": 25}
2025-05-29T17:54:47.546Z info [email protected]/memorylimiter.go:74 Memory limiter configured {"resource": {}, "otelcol.component.kind": "processor", "limit_mib": 12777, "spike_limit_mib": 3993, "check_interval": 5}
2025-05-29T17:54:52.551Z warn ec2/ec2.go:140 Failed to get EC2 instance ID, host metadata may be incomplete {"resource": {}, "otelcol.component.id": "datadog", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics", "component": "ec2_provider"}
john.peterson@COMP-HHVKJYJXCY opentelemetry-collector-contrib % kubectl logs daemonset-collector-dev-opentelemetry-collector-agent-zs768 --follow
2025-05-29T17:54:47.545Z info [email protected]/service.go:199 Setting up own telemetry... {"resource": {}}
2025-05-29T17:54:47.546Z info builders/builders.go:26 Development component. May change in the future. {"resource": {}, "otelcol.component.id": "debug", "otelcol.component.kind": "exporter", "otelcol.signal": "traces"}
2025-05-29T17:54:47.546Z info [email protected]/memorylimiter.go:149 Using percentage memory limiter {"resource": {}, "otelcol.component.kind": "processor", "total_memory_mib": 15972, "limit_percentage": 80, "spike_limit_percentage": 25}
2025-05-29T17:54:47.546Z info [email protected]/memorylimiter.go:74 Memory limiter configured {"resource": {}, "otelcol.component.kind": "processor", "limit_mib": 12777, "spike_limit_mib": 3993, "check_interval": 5}
2025-05-29T17:54:52.551Z warn ec2/ec2.go:140 Failed to get EC2 instance ID, host metadata may be incomplete {"resource": {}, "otelcol.component.id": "datadog", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics", "component": "ec2_provider"}Additional context
same underlying issue as #40372 but datadogexporter should be able to fix it by setting a default timeout when calling Azure provider pending any changes/fixes to the internal/metadataproviders/azure package