-
Notifications
You must be signed in to change notification settings - Fork 147
Disable instrumentation when an integration throws a MissingMethodException
#6807
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
MissingMethodException
disable instrumentationMissingMethodException
Datadog ReportBranch report: ✅ 0 Failed, 256924 Passed, 2423 Skipped, 21h 10m 20.29s Total Time |
Execution-Time Benchmarks Report ⏱️Execution-time results for samples comparing the following branches/commits: Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:
Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard. Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph). gantt
title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (6807) - mean (70ms) : 67, 73
. : milestone, 70,
master - mean (70ms) : 67, 72
. : milestone, 70,
section CallTarget+Inlining+NGEN
This PR (6807) - mean (1,010ms) : 987, 1034
. : milestone, 1010,
master - mean (1,007ms) : 984, 1030
. : milestone, 1007,
gantt
title Execution time (ms) FakeDbCommand (.NET Core 3.1)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (6807) - mean (103ms) : 101, 106
. : milestone, 103,
master - mean (103ms) : 100, 105
. : milestone, 103,
section CallTarget+Inlining+NGEN
This PR (6807) - mean (684ms) : 665, 703
. : milestone, 684,
master - mean (686ms) : 664, 708
. : milestone, 686,
gantt
title Execution time (ms) FakeDbCommand (.NET 6)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (6807) - mean (90ms) : 88, 93
. : milestone, 90,
master - mean (90ms) : 87, 92
. : milestone, 90,
section CallTarget+Inlining+NGEN
This PR (6807) - mean (643ms) : 630, 656
. : milestone, 643,
master - mean (644ms) : 630, 659
. : milestone, 644,
gantt
title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (6807) - mean (189ms) : 186, 193
. : milestone, 189,
master - mean (190ms) : 187, 193
. : milestone, 190,
section CallTarget+Inlining+NGEN
This PR (6807) - mean (1,102ms) : 1079, 1124
. : milestone, 1102,
master - mean (1,108ms) : 1071, 1144
. : milestone, 1108,
gantt
title Execution time (ms) HttpMessageHandler (.NET Core 3.1)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (6807) - mean (268ms) : 265, 272
. : milestone, 268,
master - mean (269ms) : 265, 273
. : milestone, 269,
section CallTarget+Inlining+NGEN
This PR (6807) - mean (870ms) : 840, 899
. : milestone, 870,
master - mean (872ms) : 844, 899
. : milestone, 872,
gantt
title Execution time (ms) HttpMessageHandler (.NET 6)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (6807) - mean (261ms) : 258, 265
. : milestone, 261,
master - mean (262ms) : 257, 266
. : milestone, 262,
section CallTarget+Inlining+NGEN
This PR (6807) - mean (854ms) : 822, 886
. : milestone, 854,
master - mean (856ms) : 826, 887
. : milestone, 856,
|
Benchmarks Report for tracer 🐌Benchmarks for #6807 compared to master:
The following thresholds were used for comparing the benchmark speeds:
Allocation changes below 0.5% are ignored. Benchmark detailsBenchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.ElasticsearchBenchmark - Faster 🎉 Same allocations ✔️
|
Benchmark | base/diff | Base Median (ns) | Diff Median (ns) | Modality |
---|---|---|---|---|
Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearch‑net6.0 | 1.123 | 1,313.57 | 1,169.89 |
Raw results
Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
---|---|---|---|---|---|---|---|---|---|
master | CallElasticsearch |
net6.0 | 1.31μs | 0.818ns | 3.17ns | 0.0132 | 0 | 0 | 976 B |
master | CallElasticsearch |
netcoreapp3.1 | 1.62μs | 0.63ns | 2.36ns | 0.0129 | 0 | 0 | 976 B |
master | CallElasticsearch |
net472 | 2.67μs | 1.07ns | 4.16ns | 0.158 | 0 | 0 | 995 B |
master | CallElasticsearchAsync |
net6.0 | 1.26μs | 0.863ns | 3.34ns | 0.0132 | 0 | 0 | 952 B |
master | CallElasticsearchAsync |
netcoreapp3.1 | 1.62μs | 1.17ns | 4.54ns | 0.013 | 0 | 0 | 1.02 KB |
master | CallElasticsearchAsync |
net472 | 2.6μs | 0.759ns | 2.94ns | 0.167 | 0 | 0 | 1.05 KB |
#6807 | CallElasticsearch |
net6.0 | 1.17μs | 0.329ns | 1.19ns | 0.0135 | 0 | 0 | 976 B |
#6807 | CallElasticsearch |
netcoreapp3.1 | 1.49μs | 0.542ns | 1.95ns | 0.0135 | 0 | 0 | 976 B |
#6807 | CallElasticsearch |
net472 | 2.64μs | 1.16ns | 4.48ns | 0.157 | 0 | 0 | 995 B |
#6807 | CallElasticsearchAsync |
net6.0 | 1.3μs | 0.435ns | 1.63ns | 0.0131 | 0 | 0 | 952 B |
#6807 | CallElasticsearchAsync |
netcoreapp3.1 | 1.67μs | 0.973ns | 3.77ns | 0.0141 | 0 | 0 | 1.02 KB |
#6807 | CallElasticsearchAsync |
net472 | 2.69μs | 1.52ns | 5.88ns | 0.166 | 0 | 0 | 1.05 KB |
Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
---|---|---|---|---|---|---|---|---|---|
master | ExecuteAsync |
net6.0 | 1.33μs | 0.48ns | 1.8ns | 0.0135 | 0 | 0 | 952 B |
master | ExecuteAsync |
netcoreapp3.1 | 1.69μs | 1.81ns | 6.78ns | 0.0125 | 0 | 0 | 952 B |
master | ExecuteAsync |
net472 | 1.8μs | 0.797ns | 3.09ns | 0.145 | 0 | 0 | 915 B |
#6807 | ExecuteAsync |
net6.0 | 1.35μs | 0.718ns | 2.69ns | 0.0135 | 0 | 0 | 952 B |
#6807 | ExecuteAsync |
netcoreapp3.1 | 1.58μs | 1.72ns | 6.67ns | 0.0126 | 0 | 0 | 952 B |
#6807 | ExecuteAsync |
net472 | 1.76μs | 0.658ns | 2.46ns | 0.145 | 0 | 0 | 915 B |
Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
---|---|---|---|---|---|---|---|---|---|
master | SendAsync |
net6.0 | 4.35μs | 1.68ns | 6.29ns | 0.0325 | 0 | 0 | 2.31 KB |
master | SendAsync |
netcoreapp3.1 | 5.36μs | 1.76ns | 6.59ns | 0.0373 | 0 | 0 | 2.85 KB |
master | SendAsync |
net472 | 7.5μs | 2.35ns | 9.08ns | 0.492 | 0 | 0 | 3.12 KB |
#6807 | SendAsync |
net6.0 | 4.35μs | 1.86ns | 6.96ns | 0.0328 | 0 | 0 | 2.31 KB |
#6807 | SendAsync |
netcoreapp3.1 | 5.58μs | 3.99ns | 15.5ns | 0.0389 | 0 | 0 | 2.85 KB |
#6807 | SendAsync |
net472 | 7.58μs | 2.26ns | 8.77ns | 0.493 | 0 | 0 | 3.12 KB |
Benchmarks.Trace.ILoggerBenchmark - Slower ⚠️ Same allocations ✔️
Slower ⚠️ in #6807
Benchmark
diff/base
Base Median (ns)
Diff Median (ns)
Modality
Benchmarks.Trace.ILoggerBenchmark.EnrichedLog‑netcoreapp3.1
1.118
2,212.90
2,473.08
Benchmark | diff/base | Base Median (ns) | Diff Median (ns) | Modality |
---|---|---|---|---|
Benchmarks.Trace.ILoggerBenchmark.EnrichedLog‑netcoreapp3.1 | 1.118 | 2,212.90 | 2,473.08 |
Raw results
Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
---|---|---|---|---|---|---|---|---|---|
master | EnrichedLog |
net6.0 | 1.64μs | 1.29ns | 4.99ns | 0.0239 | 0 | 0 | 1.69 KB |
master | EnrichedLog |
netcoreapp3.1 | 2.21μs | 1.26ns | 4.7ns | 0.0224 | 0 | 0 | 1.69 KB |
master | EnrichedLog |
net472 | 2.65μs | 1.59ns | 6.16ns | 0.257 | 0 | 0 | 1.62 KB |
#6807 | EnrichedLog |
net6.0 | 1.6μs | 1.07ns | 3.87ns | 0.024 | 0 | 0 | 1.69 KB |
#6807 | EnrichedLog |
netcoreapp3.1 | 2.47μs | 0.78ns | 2.92ns | 0.0222 | 0 | 0 | 1.69 KB |
#6807 | EnrichedLog |
net472 | 2.48μs | 1.31ns | 5.06ns | 0.258 | 0 | 0 | 1.62 KB |
Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
---|---|---|---|---|---|---|---|---|---|
master | EnrichedLog |
net6.0 | 111μs | 93.3ns | 349ns | 0.0556 | 0 | 0 | 4.31 KB |
master | EnrichedLog |
netcoreapp3.1 | 119μs | 195ns | 756ns | 0 | 0 | 0 | 4.3 KB |
master | EnrichedLog |
net472 | 150μs | 173ns | 670ns | 0.671 | 0.224 | 0 | 4.49 KB |
#6807 | EnrichedLog |
net6.0 | 112μs | 130ns | 503ns | 0.0557 | 0 | 0 | 4.3 KB |
#6807 | EnrichedLog |
netcoreapp3.1 | 118μs | 109ns | 421ns | 0 | 0 | 0 | 4.3 KB |
#6807 | EnrichedLog |
net472 | 149μs | 195ns | 756ns | 0.67 | 0.223 | 0 | 4.49 KB |
Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
---|---|---|---|---|---|---|---|---|---|
master | EnrichedLog |
net6.0 | 2.98μs | 0.801ns | 3ns | 0.0313 | 0 | 0 | 2.25 KB |
master | EnrichedLog |
netcoreapp3.1 | 4.05μs | 1.75ns | 6.54ns | 0.0303 | 0 | 0 | 2.25 KB |
master | EnrichedLog |
net472 | 4.79μs | 1.93ns | 6.69ns | 0.328 | 0 | 0 | 2.07 KB |
#6807 | EnrichedLog |
net6.0 | 3.14μs | 0.815ns | 3.16ns | 0.0314 | 0 | 0 | 2.25 KB |
#6807 | EnrichedLog |
netcoreapp3.1 | 4.38μs | 2.53ns | 9.49ns | 0.0303 | 0 | 0 | 2.25 KB |
#6807 | EnrichedLog |
net472 | 4.93μs | 2.09ns | 7.83ns | 0.327 | 0 | 0 | 2.07 KB |
Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
---|---|---|---|---|---|---|---|---|---|
master | SendReceive |
net6.0 | 1.41μs | 0.848ns | 3.28ns | 0.0163 | 0 | 0 | 1.14 KB |
master | SendReceive |
netcoreapp3.1 | 1.81μs | 0.676ns | 2.44ns | 0.0154 | 0 | 0 | 1.14 KB |
master | SendReceive |
net472 | 2.14μs | 1.19ns | 4.59ns | 0.183 | 0 | 0 | 1.16 KB |
#6807 | SendReceive |
net6.0 | 1.32μs | 0.781ns | 2.92ns | 0.0159 | 0 | 0 | 1.14 KB |
#6807 | SendReceive |
netcoreapp3.1 | 1.74μs | 0.76ns | 2.94ns | 0.0157 | 0 | 0 | 1.14 KB |
#6807 | SendReceive |
net472 | 2.09μs | 1.16ns | 4.5ns | 0.183 | 0 | 0 | 1.16 KB |
Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
---|---|---|---|---|---|---|---|---|---|
master | EnrichedLog |
net6.0 | 2.94μs | 1.17ns | 4.52ns | 0.022 | 0 | 0 | 1.62 KB |
master | EnrichedLog |
netcoreapp3.1 | 4.05μs | 2.11ns | 8.16ns | 0.0223 | 0 | 0 | 1.67 KB |
master | EnrichedLog |
net472 | 4.4μs | 2.11ns | 7.91ns | 0.326 | 0 | 0 | 2.07 KB |
#6807 | EnrichedLog |
net6.0 | 2.78μs | 1.01ns | 3.65ns | 0.0223 | 0 | 0 | 1.62 KB |
#6807 | EnrichedLog |
netcoreapp3.1 | 3.96μs | 4.66ns | 16.8ns | 0.0216 | 0 | 0 | 1.67 KB |
#6807 | EnrichedLog |
net472 | 4.52μs | 2.48ns | 9.61ns | 0.327 | 0 | 0 | 2.07 KB |
Benchmarks.Trace.SpanBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
---|---|---|---|---|---|---|---|---|---|
master | StartFinishSpan |
net6.0 | 402ns | 0.551ns | 2.06ns | 0.00813 | 0 | 0 | 576 B |
master | StartFinishSpan |
netcoreapp3.1 | 573ns | 0.353ns | 1.37ns | 0.00771 | 0 | 0 | 576 B |
master | StartFinishSpan |
net472 | 672ns | 0.156ns | 0.584ns | 0.0918 | 0 | 0 | 578 B |
master | StartFinishScope |
net6.0 | 550ns | 0.172ns | 0.664ns | 0.00987 | 0 | 0 | 696 B |
master | StartFinishScope |
netcoreapp3.1 | 760ns | 4.09ns | 22.7ns | 0.00931 | 0 | 0 | 696 B |
master | StartFinishScope |
net472 | 918ns | 0.441ns | 1.65ns | 0.104 | 0 | 0 | 658 B |
#6807 | StartFinishSpan |
net6.0 | 405ns | 0.119ns | 0.461ns | 0.00805 | 0 | 0 | 576 B |
#6807 | StartFinishSpan |
netcoreapp3.1 | 571ns | 0.503ns | 1.95ns | 0.00771 | 0 | 0 | 576 B |
#6807 | StartFinishSpan |
net472 | 645ns | 0.179ns | 0.693ns | 0.0917 | 0 | 0 | 578 B |
#6807 | StartFinishScope |
net6.0 | 559ns | 0.204ns | 0.788ns | 0.00974 | 0 | 0 | 696 B |
#6807 | StartFinishScope |
netcoreapp3.1 | 729ns | 0.506ns | 1.96ns | 0.00922 | 0 | 0 | 696 B |
#6807 | StartFinishScope |
net472 | 917ns | 0.809ns | 3.13ns | 0.104 | 0 | 0 | 658 B |
Benchmarks.Trace.TraceAnnotationsBenchmark - Slower ⚠️ Same allocations ✔️
Slower ⚠️ in #6807
Benchmark
diff/base
Base Median (ns)
Diff Median (ns)
Modality
Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑net6.0
1.119
601.25
672.61
Benchmark | diff/base | Base Median (ns) | Diff Median (ns) | Modality |
---|---|---|---|---|
Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑net6.0 | 1.119 | 601.25 | 672.61 |
Raw results
Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
---|---|---|---|---|---|---|---|---|---|
master | RunOnMethodBegin |
net6.0 | 601ns | 0.204ns | 0.79ns | 0.0099 | 0 | 0 | 696 B |
master | RunOnMethodBegin |
netcoreapp3.1 | 966ns | 0.582ns | 2.25ns | 0.00951 | 0 | 0 | 696 B |
master | RunOnMethodBegin |
net472 | 1.14μs | 0.189ns | 0.706ns | 0.104 | 0 | 0 | 658 B |
#6807 | RunOnMethodBegin |
net6.0 | 672ns | 0.248ns | 0.926ns | 0.00975 | 0 | 0 | 696 B |
#6807 | RunOnMethodBegin |
netcoreapp3.1 | 961ns | 0.743ns | 2.88ns | 0.00929 | 0 | 0 | 696 B |
#6807 | RunOnMethodBegin |
net472 | 1.15μs | 0.646ns | 2.5ns | 0.104 | 0 | 0 | 658 B |
tracer/src/Datadog.Trace/ClrProfiler/CallTarget/Handlers/IntegrationOptions.cs
Outdated
Show resolved
Hide resolved
25fac13
to
d2fa8c8
Compare
…rationOptions.cs Co-authored-by: Steven Bouwkamp <[email protected]>
d2fa8c8
to
62ab6b4
Compare
## Summary of changes revert the revert of #6792 The missingMethodExceptions were most likely caused by a trimmed app ([see discussion](https://dd.slack.com/archives/C07TBLYDQHX/p1741907211362359)). We had some defense against this for _instrumented_ types, but not for duck-typed types, and this is where the issue laid. This was taken care of in #6807 for the generic case, and also after [talking with DSM](https://dd.slack.com/archives/C069TJPTFPG/p1743500681812779), they agree that we should try to instrument only user-defined protobufs, so I excluded the microsoft protobufs (that were causing the issue). Thus, this PR should be fine to merge as-is, even though it's best to merge it after #6807 for maximum safety. ## Reason for change ## Implementation details ## Test coverage ## Other details <!-- Fixes #{issue} --> <!--⚠️ Note: where possible, please obtain 2 approvals prior to merging. Unless CODEOWNERS specifies otherwise, for external teams it is typically best to have one review from a team member, and one review from apm-dotnet. Trivial changes do not require 2 reviews. -->
Summary of changes
When an integration methods throws a
MissingMethodException
orMissingFieldException
, disable the integration.Reason for change
This scenario is most likely to occur when the app is trimmed (plus in some relatively obscure scenarios with binding redirects or strange deployments). In those cases, the exception will likely always be thrown, which could cause longer term issues. To fix, the customer should enable
Implementation details
Just as we do for "fatal"
DuckTypeException
orCallTargetInvokerExecption
, disable the integration that generates the exception when this occurs.Test coverage
I realised we don't have any tests (unit or integration) for the
DuckTypeException
behaviour, so added some, and added some for this behaviour too.Other details
One slight issue came to mind. If we create a scope inside the
OnBegin
part of an integration, and then we hit this error, then we never close the scope, and will start leaking. I wonder if we could/should look to avoid that.In the past, we only expected exceptions in the method when there were programming errors, so they should be relatively rare. But for trimming it's a different scenario, that's dependent on the customer, so maybe that's an issue?