Skip to content

[Test Package Versions Bump] Updating package versions #6732

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Mar 20, 2025

Conversation

github-actions[bot]
Copy link
Contributor

@github-actions github-actions bot commented Mar 2, 2025

Updates the package versions for integration tests.

@github-actions github-actions bot requested review from a team as code owners March 2, 2025 00:26
@datadog-datadog-prod-us1
Copy link

datadog-datadog-prod-us1 bot commented Mar 2, 2025

Datadog Report

Branch report: bot/test-package-versions-bump
Commit report: afb4173
Test service: dd-trace-dotnet

✅ 0 Failed, 248746 Passed, 2407 Skipped, 20h 47m 27.48s Total Time
⌛ 1 Performance Regression

⌛ Performance Regressions vs Default Branch (1)

  • Baseline_cpu - scenarios 2.78s (+478.74ms, +21%) - Details

@andrewlock
Copy link
Member

andrewlock commented Mar 2, 2025

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

  • Welch test with statistical test for significance of 5%
  • Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6732) - mean (69ms)  : 67, 72
     .   : milestone, 69,
    master - mean (69ms)  : 67, 71
     .   : milestone, 69,

    section CallTarget+Inlining+NGEN
    This PR (6732) - mean (1,010ms)  : 985, 1034
     .   : milestone, 1010,
    master - mean (1,006ms)  : 977, 1034
     .   : milestone, 1006,

Loading
gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6732) - mean (102ms)  : 100, 105
     .   : milestone, 102,
    master - mean (102ms)  : 99, 105
     .   : milestone, 102,

    section CallTarget+Inlining+NGEN
    This PR (6732) - mean (688ms)  : 674, 702
     .   : milestone, 688,
    master - mean (685ms)  : 669, 702
     .   : milestone, 685,

Loading
gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6732) - mean (89ms)  : 88, 91
     .   : milestone, 89,
    master - mean (89ms)  : 86, 91
     .   : milestone, 89,

    section CallTarget+Inlining+NGEN
    This PR (6732) - mean (641ms)  : 624, 658
     .   : milestone, 641,
    master - mean (642ms)  : 622, 662
     .   : milestone, 642,

Loading
gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6732) - mean (190ms)  : 185, 195
     .   : milestone, 190,
    master - mean (191ms)  : 187, 194
     .   : milestone, 191,

    section CallTarget+Inlining+NGEN
    This PR (6732) - mean (1,111ms)  : 1078, 1144
     .   : milestone, 1111,
    master - mean (1,107ms)  : 1085, 1130
     .   : milestone, 1107,

Loading
gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6732) - mean (270ms)  : 266, 273
     .   : milestone, 270,
    master - mean (269ms)  : 265, 273
     .   : milestone, 269,

    section CallTarget+Inlining+NGEN
    This PR (6732) - mean (880ms)  : 846, 914
     .   : milestone, 880,
    master - mean (876ms)  : 839, 912
     .   : milestone, 876,

Loading
gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6732) - mean (262ms)  : 259, 265
     .   : milestone, 262,
    master - mean (262ms)  : 258, 265
     .   : milestone, 262,

    section CallTarget+Inlining+NGEN
    This PR (6732) - mean (860ms)  : 831, 889
     .   : milestone, 860,
    master - mean (859ms)  : 828, 890
     .   : milestone, 859,

Loading

@andrewlock
Copy link
Member

andrewlock commented Mar 2, 2025

Benchmarks Report for tracer 🐌

Benchmarks for #6732 compared to master:

  • All benchmarks have the same speed
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartStopWithChild net6.0 7.92μs 45.1ns 316ns 0.0192 0.00769 0 5.61 KB
master StartStopWithChild netcoreapp3.1 9.91μs 52.9ns 280ns 0.0237 0.00947 0 5.8 KB
master StartStopWithChild net472 16.1μs 47.9ns 179ns 1.04 0.313 0.0963 6.2 KB
#6732 StartStopWithChild net6.0 7.86μs 41.4ns 227ns 0.0165 0.00826 0 5.61 KB
#6732 StartStopWithChild netcoreapp3.1 9.97μs 54.4ns 308ns 0.0203 0.0101 0 5.8 KB
#6732 StartStopWithChild net472 16.2μs 48.1ns 186ns 1.04 0.296 0.096 6.21 KB
Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 496μs 605ns 2.34μs 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 654μs 425ns 1.64μs 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces net472 854μs 1.56μs 6.06μs 0.425 0 0 3.3 KB
#6732 WriteAndFlushEnrichedTraces net6.0 500μs 636ns 2.38μs 0 0 0 2.7 KB
#6732 WriteAndFlushEnrichedTraces netcoreapp3.1 660μs 492ns 1.91μs 0 0 0 2.7 KB
#6732 WriteAndFlushEnrichedTraces net472 845μs 787ns 3.05μs 0.419 0 0 3.3 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net6.0 128μs 359ns 1.39μs 0.191 0 0 14.47 KB
master SendRequest netcoreapp3.1 143μs 508ns 1.97μs 0.215 0 0 17.27 KB
master SendRequest net472 0.000163ns 9.56E‑05ns 0.000358ns 0 0 0 0 b
#6732 SendRequest net6.0 134μs 675ns 4μs 0.13 0 0 14.47 KB
#6732 SendRequest netcoreapp3.1 149μs 230ns 893ns 0.224 0 0 17.27 KB
#6732 SendRequest net472 0.000819ns 0.000314ns 0.00117ns 0 0 0 0 b
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 627μs 4.17μs 41.3μs 0.571 0 0 41.62 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 732μs 4.51μs 44.2μs 0.334 0 0 41.87 KB
master WriteAndFlushEnrichedTraces net472 855μs 4.34μs 20.8μs 8.08 2.55 0.425 53.32 KB
#6732 WriteAndFlushEnrichedTraces net6.0 566μs 2.99μs 14.3μs 0.558 0 0 41.67 KB
#6732 WriteAndFlushEnrichedTraces netcoreapp3.1 706μs 4.09μs 37.3μs 0.332 0 0 41.81 KB
#6732 WriteAndFlushEnrichedTraces net472 838μs 3.64μs 13.6μs 8.45 2.53 0.422 53.27 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net6.0 1.26μs 1.45ns 5.6ns 0.0145 0 0 1.02 KB
master ExecuteNonQuery netcoreapp3.1 1.71μs 2.1ns 8.14ns 0.0137 0 0 1.02 KB
master ExecuteNonQuery net472 2.08μs 4.28ns 16.6ns 0.157 0.00104 0 987 B
#6732 ExecuteNonQuery net6.0 1.36μs 0.983ns 3.68ns 0.0143 0 0 1.02 KB
#6732 ExecuteNonQuery netcoreapp3.1 1.79μs 0.888ns 3.44ns 0.0134 0 0 1.02 KB
#6732 ExecuteNonQuery net472 2.02μs 1.81ns 7ns 0.156 0.00102 0 987 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net6.0 1.35μs 0.414ns 1.6ns 0.0136 0 0 976 B
master CallElasticsearch netcoreapp3.1 1.66μs 1.68ns 6.51ns 0.0132 0 0 976 B
master CallElasticsearch net472 2.72μs 6.62ns 24.8ns 0.157 0 0 995 B
master CallElasticsearchAsync net6.0 1.25μs 0.452ns 1.69ns 0.0131 0 0 952 B
master CallElasticsearchAsync netcoreapp3.1 1.64μs 0.872ns 3.38ns 0.014 0 0 1.02 KB
master CallElasticsearchAsync net472 2.61μs 1.9ns 7.36ns 0.167 0 0 1.05 KB
#6732 CallElasticsearch net6.0 1.24μs 1.58ns 5.91ns 0.0138 0 0 976 B
#6732 CallElasticsearch netcoreapp3.1 1.59μs 2.42ns 9.39ns 0.0126 0 0 976 B
#6732 CallElasticsearch net472 2.52μs 2.77ns 10ns 0.157 0 0 995 B
#6732 CallElasticsearchAsync net6.0 1.36μs 0.664ns 2.39ns 0.0135 0 0 952 B
#6732 CallElasticsearchAsync netcoreapp3.1 1.66μs 0.563ns 2.03ns 0.014 0 0 1.02 KB
#6732 CallElasticsearchAsync net472 2.78μs 1.41ns 5.46ns 0.167 0 0 1.05 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net6.0 1.36μs 0.462ns 1.73ns 0.013 0 0 952 B
master ExecuteAsync netcoreapp3.1 1.63μs 1.32ns 4.93ns 0.0122 0 0 952 B
master ExecuteAsync net472 1.74μs 0.494ns 1.85ns 0.145 0 0 915 B
#6732 ExecuteAsync net6.0 1.25μs 0.311ns 1.16ns 0.0131 0 0 952 B
#6732 ExecuteAsync netcoreapp3.1 1.62μs 0.494ns 1.85ns 0.0122 0 0 952 B
#6732 ExecuteAsync net472 1.8μs 1.24ns 4.65ns 0.145 0 0 915 B
Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net6.0 4.45μs 3.64ns 13.6ns 0.0309 0 0 2.31 KB
master SendAsync netcoreapp3.1 5.45μs 5.27ns 20.4ns 0.0381 0 0 2.85 KB
master SendAsync net472 7.4μs 2.02ns 7ns 0.494 0 0 3.12 KB
#6732 SendAsync net6.0 4.31μs 2.74ns 9.86ns 0.0323 0 0 2.31 KB
#6732 SendAsync netcoreapp3.1 5.27μs 5.96ns 23.1ns 0.0395 0 0 2.85 KB
#6732 SendAsync net472 7.43μs 2.76ns 10.3ns 0.495 0 0 3.12 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 1.57μs 0.497ns 1.86ns 0.0236 0 0 1.64 KB
master EnrichedLog netcoreapp3.1 2.24μs 1.43ns 5.37ns 0.0224 0 0 1.64 KB
master EnrichedLog net472 2.58μs 0.624ns 2.25ns 0.25 0 0 1.57 KB
#6732 EnrichedLog net6.0 1.49μs 2.18ns 8.16ns 0.023 0 0 1.64 KB
#6732 EnrichedLog netcoreapp3.1 2.21μs 2.03ns 7.58ns 0.0221 0 0 1.64 KB
#6732 EnrichedLog net472 2.46μs 0.709ns 2.46ns 0.249 0 0 1.57 KB
Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 111μs 143ns 553ns 0.0555 0 0 4.28 KB
master EnrichedLog netcoreapp3.1 117μs 146ns 547ns 0.0588 0 0 4.28 KB
master EnrichedLog net472 149μs 223ns 863ns 0.678 0.226 0 4.46 KB
#6732 EnrichedLog net6.0 113μs 132ns 511ns 0.056 0 0 4.28 KB
#6732 EnrichedLog netcoreapp3.1 116μs 143ns 515ns 0 0 0 4.28 KB
#6732 EnrichedLog net472 151μs 109ns 422ns 0.679 0.226 0 4.46 KB
Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 3.08μs 1.22ns 4.57ns 0.0308 0 0 2.2 KB
master EnrichedLog netcoreapp3.1 4.31μs 4.52ns 17.5ns 0.0304 0 0 2.2 KB
master EnrichedLog net472 4.71μs 1.77ns 6.87ns 0.32 0 0 2.02 KB
#6732 EnrichedLog net6.0 3.06μs 0.841ns 3.26ns 0.0307 0 0 2.2 KB
#6732 EnrichedLog netcoreapp3.1 4.15μs 1.06ns 4.12ns 0.0299 0 0 2.2 KB
#6732 EnrichedLog net472 4.69μs 1.4ns 5.44ns 0.321 0 0 2.02 KB
Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net6.0 1.38μs 0.903ns 3.5ns 0.0158 0 0 1.14 KB
master SendReceive netcoreapp3.1 1.81μs 0.735ns 2.85ns 0.0153 0 0 1.14 KB
master SendReceive net472 2.12μs 1.37ns 5.31ns 0.183 0 0 1.16 KB
#6732 SendReceive net6.0 1.37μs 0.873ns 3.38ns 0.0163 0 0 1.14 KB
#6732 SendReceive netcoreapp3.1 1.79μs 0.892ns 3.45ns 0.0156 0 0 1.14 KB
#6732 SendReceive net472 2.1μs 5.18ns 20.1ns 0.183 0 0 1.16 KB
Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.82μs 1.27ns 4.74ns 0.0225 0 0 1.6 KB
master EnrichedLog netcoreapp3.1 4.05μs 1.57ns 5.88ns 0.0221 0 0 1.65 KB
master EnrichedLog net472 4.42μs 4.31ns 16.7ns 0.323 0 0 2.04 KB
#6732 EnrichedLog net6.0 2.69μs 0.966ns 3.74ns 0.0216 0 0 1.6 KB
#6732 EnrichedLog netcoreapp3.1 3.96μs 0.792ns 2.97ns 0.0214 0 0 1.65 KB
#6732 EnrichedLog net472 4.38μs 1.86ns 6.72ns 0.323 0 0 2.04 KB
Benchmarks.Trace.SpanBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net6.0 402ns 0.0703ns 0.263ns 0.00801 0 0 576 B
master StartFinishSpan netcoreapp3.1 577ns 0.311ns 1.2ns 0.00777 0 0 576 B
master StartFinishSpan net472 665ns 0.277ns 1.03ns 0.0915 0 0 578 B
master StartFinishScope net6.0 477ns 0.103ns 0.398ns 0.00974 0 0 696 B
master StartFinishScope netcoreapp3.1 742ns 0.362ns 1.4ns 0.00919 0 0 696 B
master StartFinishScope net472 843ns 0.172ns 0.667ns 0.104 0 0 658 B
#6732 StartFinishSpan net6.0 396ns 0.071ns 0.275ns 0.00811 0 0 576 B
#6732 StartFinishSpan netcoreapp3.1 618ns 0.383ns 1.48ns 0.00786 0 0 576 B
#6732 StartFinishSpan net472 651ns 0.166ns 0.622ns 0.0916 0 0 578 B
#6732 StartFinishScope net6.0 496ns 0.111ns 0.416ns 0.00979 0 0 696 B
#6732 StartFinishScope netcoreapp3.1 696ns 0.388ns 1.5ns 0.0094 0 0 696 B
#6732 StartFinishScope net472 823ns 0.321ns 1.24ns 0.105 0 0 658 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net6.0 639ns 0.122ns 0.438ns 0.00957 0 0 696 B
master RunOnMethodBegin netcoreapp3.1 922ns 0.391ns 1.41ns 0.0093 0 0 696 B
master RunOnMethodBegin net472 1.12μs 0.367ns 1.42ns 0.104 0 0 658 B
#6732 RunOnMethodBegin net6.0 666ns 0.499ns 1.93ns 0.00966 0 0 696 B
#6732 RunOnMethodBegin netcoreapp3.1 968ns 0.523ns 2.02ns 0.00921 0 0 696 B
#6732 RunOnMethodBegin net472 1.11μs 0.447ns 1.73ns 0.104 0 0 658 B

@@ -354,13 +354,13 @@

<!-- Integration: xunit.v3.core -->
<!-- Assembly: xunit.v3.core -->
<!-- Latest package https://www.nuget.org/packages/xunit.v3/1.1.0 -->
<!-- Latest package https://www.nuget.org/packages/xunit.v3/2.0.0 -->
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will let CI Vis know

<!-- Latest package https://www.nuget.org/packages/IBMMQDotnetClient/9.4.1 -->
<PackageReference Include="IBMMQDotnetClient" Version="9.4.1" />
<!-- Latest package https://www.nuget.org/packages/IBMMQDotnetClient/9.4.2 -->
<PackageReference Include="IBMMQDotnetClient" Version="9.4.2" />
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Think will need to look into what changed here as we don't run these on CI

<!-- Latest package https://www.nuget.org/packages/Microsoft.Azure.Cosmos/3.47.1 -->
<PackageReference Include="Microsoft.Azure.Cosmos" Version="3.47.1" />
<!-- Latest package https://www.nuget.org/packages/Microsoft.Azure.Cosmos/3.47.2 -->
<PackageReference Include="Microsoft.Azure.Cosmos" Version="3.47.2" />
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will need to test this out locally.

@github-actions github-actions bot force-pushed the bot/test-package-versions-bump branch from c796646 to 094b706 Compare March 9, 2025 00:23
@github-actions github-actions bot force-pushed the bot/test-package-versions-bump branch from 094b706 to a2a9a6c Compare March 16, 2025 00:27
andrewlock added a commit that referenced this pull request Mar 18, 2025
## Summary of changes

Fixes the Microsoft.Extensions.Telemetry integration for 9.3.0

## Reason for change

9.3.0 [introduced a new optional
parameter](dotnet/extensions#5574) to a public
method we instrument, breaking our integration

## Implementation details

Add instrumentation for the new signature

## Test coverage

This is the test - if everything passes, we're good

## Other details

Identified in #6732
@github-actions github-actions bot force-pushed the bot/test-package-versions-bump branch from a2a9a6c to afb4173 Compare March 18, 2025 13:03
Copy link
Member

@andrewlock andrewlock left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM now

@andrewlock andrewlock merged commit 9ae2cf5 into master Mar 20, 2025
119 of 121 checks passed
@andrewlock andrewlock deleted the bot/test-package-versions-bump branch March 20, 2025 09:54
@github-actions github-actions bot added this to the vNext-v3 milestone Mar 20, 2025
@bouwkast bouwkast added area:test-apps apps used to test integrations dependencies Pull requests that update a dependency file area:dependabot dependabot updates labels Mar 24, 2025
ddyurchenko pushed a commit that referenced this pull request Mar 31, 2025
## Summary of changes

Fixes the Microsoft.Extensions.Telemetry integration for 9.3.0

## Reason for change

9.3.0 [introduced a new optional
parameter](dotnet/extensions#5574) to a public
method we instrument, breaking our integration

## Implementation details

Add instrumentation for the new signature

## Test coverage

This is the test - if everything passes, we're good

## Other details

Identified in #6732
ddyurchenko pushed a commit that referenced this pull request Mar 31, 2025
Updates the package versions for integration tests.

Co-authored-by: andrewlock <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:dependabot dependabot updates area:test-apps apps used to test integrations dependencies Pull requests that update a dependency file
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants