Skip to content

Fayssal/update bp infra #6924

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 18 commits into from
May 6, 2025
Merged

Fayssal/update bp infra #6924

merged 18 commits into from
May 6, 2025

Conversation

faydef
Copy link
Contributor

@faydef faydef commented May 5, 2025

Summary of changes

Enables the following by updating the BP infra:

  • Running microbenchmarks with dotnet 9.0.203
  • Running Linux macrobenchmarks with dotnet 9.0.203
  • Running Windows microbenchmarks with dotnet 9.0.203

Reason for change

The benchmark runs have been failing since the dotnet SDK version has been updated

Implementation details

  • Rebuilt the Linux macrobenchmark image in order to use the latest version of the dotnet SDK
  • Changed and rebuilt the macrobenchmark AMI
  • Adapted the microbenchmark execution

Test coverage

Other details

More documentation on Benchmarking platform
More Windows specific doc

@faydef faydef requested a review from a team as a code owner May 5, 2025 08:18
@faydef
Revert "disable artifacts"
69d2ca6 
This reverts commit e102047.
@datadog-datadog-prod-us1
Copy link

datadog-datadog-prod-us1 bot commented May 5, 2025 •
edited
Loading

Datadog Report

Branch report: fayssal/update-bp-infra
Commit report: 69d2ca6
Test service: dd-trace-dotnet

✅ 0 Failed, 249689 Passed, 2263 Skipped, 18h 1m 43.74s Total Time

@pr-commenter
Copy link

pr-commenter bot commented May 5, 2025

Benchmarks

Benchmarks Report for tracer 🐌

Benchmarks for #6924 compared to master:

  • 1 benchmarks are slower, with geometric mean 1.134
  • 3 benchmarks have fewer allocations
  • 1 benchmarks have more allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #6924

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.ActivityBenchmark.StartStopWithChild‑net472 6.17 KB 6.24 KB 69 B 1.12%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartStopWithChild net6.0 10.4μs 58ns 371ns 0 0 0 5.57 KB
master StartStopWithChild netcoreapp3.1 14.5μs 59.3ns 230ns 0 0 0 5.76 KB
master StartStopWithChild net472 21.1μs 116ns 688ns 0.89 0.222 0 6.17 KB
#6924 StartStopWithChild net6.0 10.3μs 57.8ns 370ns 0 0 0 5.57 KB
#6924 StartStopWithChild netcoreapp3.1 12.9μs 70ns 371ns 0 0 0 5.77 KB
#6924 StartStopWithChild net472 22.1μs 124ns 814ns 0.982 0.327 0 6.24 KB
Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Fewer allocations 🎉

Fewer allocations 🎉 in #6924

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.AgentWriterBenchmark.WriteAndFlushEnrichedTraces‑net472 3.35 KB 3.31 KB -46 B -1.37%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 958μs 73.4ns 275ns 0 0 0 2.71 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 1.02ms 366ns 1.32μs 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces net472 1.22ms 2.93μs 11.4μs 0 0 0 3.35 KB
#6924 WriteAndFlushEnrichedTraces net6.0 968μs 606ns 2.35μs 0 0 0 2.7 KB
#6924 WriteAndFlushEnrichedTraces netcoreapp3.1 1.04ms 126ns 486ns 0 0 0 2.7 KB
#6924 WriteAndFlushEnrichedTraces net472 1.25ms 42.8ns 154ns 0 0 0 3.31 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net6.0 61.2μs 34.8ns 121ns 0 0 0 14.53 KB
master SendRequest netcoreapp3.1 71.1μs 69.2ns 249ns 0 0 0 17.42 KB
master SendRequest net472 0.00968ns 0.00292ns 0.0113ns 0 0 0 0 b
#6924 SendRequest net6.0 60.9μs 77.5ns 290ns 0 0 0 14.53 KB
#6924 SendRequest netcoreapp3.1 71.2μs 100ns 389ns 0 0 0 17.42 KB
#6924 SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Slower ⚠️ Fewer allocations 🎉

Slower ⚠️ in #6924

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑netcoreapp3.1 1.134 644,834.18 731,120.31 bimodal

Fewer allocations 🎉 in #6924

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑net472 56.3 KB 55.79 KB -511 B -0.91%
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑netcoreapp3.1 42.28 KB 41.83 KB -451 B -1.07%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 708μs 3.66μs 18.7μs 0 0 0 41.67 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 640μs 2.98μs 11.9μs 0 0 0 42.28 KB
master WriteAndFlushEnrichedTraces net472 952μs 3.91μs 15.2μs 4.81 0 0 56.3 KB
#6924 WriteAndFlushEnrichedTraces net6.0 752μs 708ns 2.65μs 0 0 0 41.63 KB
#6924 WriteAndFlushEnrichedTraces netcoreapp3.1 741μs 4.28μs 33.4μs 0 0 0 41.83 KB
#6924 WriteAndFlushEnrichedTraces net472 912μs 3.38μs 13.1μs 4.46 0 0 55.79 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net6.0 1.97μs 4.5ns 17.4ns 0 0 0 1.03 KB
master ExecuteNonQuery netcoreapp3.1 2.43μs 6.55ns 25.4ns 0 0 0 1.02 KB
master ExecuteNonQuery net472 2.78μs 4.4ns 17ns 0.152 0.0138 0 995 B
#6924 ExecuteNonQuery net6.0 1.99μs 7.32ns 27.4ns 0 0 0 1.03 KB
#6924 ExecuteNonQuery netcoreapp3.1 2.48μs 12ns 46.4ns 0 0 0 1.02 KB
#6924 ExecuteNonQuery net472 2.76μs 3.35ns 13ns 0.15 0.0137 0 995 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net6.0 1.78μs 8.35ns 31.2ns 0 0 0 1.04 KB
master CallElasticsearch netcoreapp3.1 2.23μs 10.8ns 47.2ns 0 0 0 1.04 KB
master CallElasticsearch net472 3.56μs 2.44ns 9.44ns 0.16 0 0 1.05 KB
master CallElasticsearchAsync net6.0 1.84μs 8.75ns 36.1ns 0 0 0 1.02 KB
master CallElasticsearchAsync netcoreapp3.1 2.33μs 6.56ns 25.4ns 0 0 0 1.09 KB
master CallElasticsearchAsync net472 3.67μs 2.45ns 9.16ns 0.165 0 0 1.11 KB
#6924 CallElasticsearch net6.0 1.75μs 8.84ns 37.5ns 0 0 0 1.04 KB
#6924 CallElasticsearch netcoreapp3.1 2.44μs 11.3ns 46.7ns 0 0 0 1.04 KB
#6924 CallElasticsearch net472 3.57μs 1.73ns 6.71ns 0.162 0 0 1.05 KB
#6924 CallElasticsearchAsync net6.0 1.83μs 8.71ns 36.9ns 0 0 0 1.02 KB
#6924 CallElasticsearchAsync netcoreapp3.1 2.34μs 11.3ns 43.6ns 0 0 0 1.09 KB
#6924 CallElasticsearchAsync net472 3.56μs 2.74ns 10.6ns 0.16 0 0 1.11 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net6.0 1.79μs 5.09ns 19.7ns 0 0 0 960 B
master ExecuteAsync netcoreapp3.1 2.26μs 11.3ns 48.1ns 0 0 0 960 B
master ExecuteAsync net472 2.57μs 2.91ns 11.3ns 0.14 0 0 923 B
#6924 ExecuteAsync net6.0 1.86μs 0.712ns 2.76ns 0 0 0 960 B
#6924 ExecuteAsync netcoreapp3.1 2.33μs 10.3ns 39.9ns 0 0 0 960 B
#6924 ExecuteAsync net472 2.5μs 1.58ns 6.12ns 0.138 0 0 923 B
Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net6.0 7.07μs 17.7ns 68.7ns 0 0 0 2.37 KB
master SendAsync netcoreapp3.1 8.28μs 15.2ns 58.7ns 0 0 0 2.9 KB
master SendAsync net472 12.5μs 10.5ns 39.4ns 0.499 0 0 3.19 KB
#6924 SendAsync net6.0 6.85μs 12.7ns 49.2ns 0 0 0 2.37 KB
#6924 SendAsync netcoreapp3.1 8.58μs 23.4ns 90.7ns 0 0 0 2.9 KB
#6924 SendAsync net472 12.7μs 18.3ns 70.9ns 0.505 0 0 3.19 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.56μs 1.05ns 3.94ns 0 0 0 1.76 KB
master EnrichedLog netcoreapp3.1 3.43μs 15.3ns 59.2ns 0 0 0 1.76 KB
master EnrichedLog net472 4.01μs 3.5ns 13.6ns 0.262 0 0 1.69 KB
#6924 EnrichedLog net6.0 2.62μs 13.3ns 51.4ns 0 0 0 1.76 KB
#6924 EnrichedLog netcoreapp3.1 3.52μs 17.7ns 81.1ns 0 0 0 1.76 KB
#6924 EnrichedLog net472 3.9μs 3.77ns 13.6ns 0.253 0 0 1.69 KB
Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 123μs 202ns 757ns 0 0 0 4.37 KB
master EnrichedLog netcoreapp3.1 128μs 66.5ns 230ns 0 0 0 4.37 KB
master EnrichedLog net472 169μs 162ns 627ns 0 0 0 4.57 KB
#6924 EnrichedLog net6.0 123μs 64.5ns 241ns 0 0 0 4.37 KB
#6924 EnrichedLog netcoreapp3.1 128μs 210ns 813ns 0 0 0 4.37 KB
#6924 EnrichedLog net472 166μs 155ns 599ns 0 0 0 4.57 KB
Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 5.14μs 8.39ns 32.5ns 0 0 0 2.32 KB
master EnrichedLog netcoreapp3.1 6.52μs 21.2ns 82ns 0 0 0 2.32 KB
master EnrichedLog net472 7.45μs 5.05ns 18.9ns 0.334 0 0 2.14 KB
#6924 EnrichedLog net6.0 5.11μs 20.3ns 76ns 0 0 0 2.32 KB
#6924 EnrichedLog netcoreapp3.1 6.59μs 20.7ns 80.1ns 0 0 0 2.32 KB
#6924 EnrichedLog net472 7.27μs 10.9ns 42.4ns 0.327 0 0 2.14 KB
Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net6.0 1.99μs 5.91ns 22.9ns 0 0 0 1.21 KB
master SendReceive netcoreapp3.1 2.62μs 7.06ns 27.3ns 0 0 0 1.21 KB
master SendReceive net472 3.06μs 2.36ns 8.83ns 0.186 0 0 1.21 KB
#6924 SendReceive net6.0 2.01μs 9.45ns 36.6ns 0 0 0 1.21 KB
#6924 SendReceive netcoreapp3.1 2.71μs 12.1ns 46.8ns 0 0 0 1.21 KB
#6924 SendReceive net472 3.05μs 0.796ns 3.08ns 0.183 0 0 1.21 KB
Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 4.23μs 7.49ns 29ns 0 0 0 1.64 KB
master EnrichedLog netcoreapp3.1 5.57μs 16.3ns 63.1ns 0 0 0 1.69 KB
master EnrichedLog net472 6.76μs 9.4ns 36.4ns 0.304 0 0 2.08 KB
#6924 EnrichedLog net6.0 4.4μs 13.2ns 51.1ns 0 0 0 1.64 KB
#6924 EnrichedLog netcoreapp3.1 5.65μs 11ns 42.6ns 0 0 0 1.69 KB
#6924 EnrichedLog net472 6.6μs 8.16ns 31.6ns 0.298 0 0 2.08 KB
Benchmarks.Trace.SpanBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net6.0 757ns 3.81ns 16.2ns 0 0 0 584 B
master StartFinishSpan netcoreapp3.1 955ns 4.57ns 18.9ns 0 0 0 584 B
master StartFinishSpan net472 931ns 0.555ns 1.92ns 0.093 0 0 586 B
master StartFinishScope net6.0 915ns 4.67ns 20.9ns 0 0 0 704 B
master StartFinishScope netcoreapp3.1 1.19μs 5.9ns 25ns 0 0 0 704 B
master StartFinishScope net472 1.11μs 0.198ns 0.712ns 0.106 0 0 666 B
#6924 StartFinishSpan net6.0 751ns 3.9ns 19.9ns 0 0 0 584 B
#6924 StartFinishSpan netcoreapp3.1 932ns 4.86ns 24.3ns 0 0 0 584 B
#6924 StartFinishSpan net472 936ns 0.259ns 0.969ns 0.0889 0 0 586 B
#6924 StartFinishScope net6.0 910ns 3.97ns 15.4ns 0 0 0 704 B
#6924 StartFinishScope netcoreapp3.1 1.17μs 2.08ns 8.04ns 0 0 0 704 B
#6924 StartFinishScope net472 1.11μs 0.166ns 0.598ns 0.1 0 0 666 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net6.0 1.03μs 1.7ns 6.59ns 0 0 0 704 B
master RunOnMethodBegin netcoreapp3.1 1.32μs 6.91ns 34.6ns 0 0 0 704 B
master RunOnMethodBegin net472 1.4μs 1.19ns 4.6ns 0.105 0 0 666 B
#6924 RunOnMethodBegin net6.0 1.02μs 5.55ns 33.3ns 0 0 0 704 B
#6924 RunOnMethodBegin netcoreapp3.1 1.35μs 7.01ns 32.1ns 0 0 0 704 B
#6924 RunOnMethodBegin net472 1.38μs 1.06ns 3.96ns 0.104 0 0 666 B

@andrewlock
Copy link
Member

Benchmarks Report for tracer 🐌

Benchmarks for #6924 compared to master:

  • 1 benchmarks are faster, with geometric mean 2.052
  • 1 benchmarks have fewer allocations
  • 1 benchmarks have more allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #6924

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.ActivityBenchmark.StartStopWithChild‑net6.0 5.56 KB 5.6 KB 38 B 0.68%

Fewer allocations 🎉 in #6924

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.ActivityBenchmark.StartStopWithChild‑net472 6.19 KB 6.13 KB -60 B -0.97%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartStopWithChild net6.0 8.11μs 141ns 1.41μs 0 0 0 5.56 KB
master StartStopWithChild netcoreapp3.1 10.5μs 125ns 1.25μs 0 0 0 5.72 KB
master StartStopWithChild net472 15.3μs 87.5ns 649ns 0.929 0.214 0 6.19 KB
#6924 StartStopWithChild net6.0 8.44μs 238ns 2.34μs 0 0 0 5.6 KB
#6924 StartStopWithChild netcoreapp3.1 10.3μs 106ns 1.05μs 0 0 0 5.74 KB
#6924 StartStopWithChild net472 15.4μs 87.9ns 664ns 1.01 0.312 0.078 6.13 KB
Benchmarks.Trace.AgentWriterBenchmark - Faster 🎉 Same allocations ✔️

Faster 🎉 in #6924

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.AgentWriterBenchmark.WriteAndFlushEnrichedTraces‑net6.0 2.052 1,046,306.25 509,889.62

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 1.04ms 5.39μs 25.8μs 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 638μs 605ns 2.18μs 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces net472 847μs 500ns 1.8μs 0 0 0 3.31 KB
#6924 WriteAndFlushEnrichedTraces net6.0 510μs 554ns 1.92μs 0 0 0 2.7 KB
#6924 WriteAndFlushEnrichedTraces netcoreapp3.1 652μs 949ns 3.42μs 0 0 0 2.7 KB
#6924 WriteAndFlushEnrichedTraces net472 865μs 591ns 2.29μs 0 0 0 3.31 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net6.0 127μs 725ns 5.76μs 0 0 0 14.48 KB
master SendRequest netcoreapp3.1 148μs 508ns 1.97μs 0 0 0 17.28 KB
master SendRequest net472 0.00266ns 0.00105ns 0.00406ns 0 0 0 0 b
#6924 SendRequest net6.0 132μs 771ns 7.03μs 0 0 0 14.48 KB
#6924 SendRequest netcoreapp3.1 152μs 725ns 3.08μs 0 0 0 17.28 KB
#6924 SendRequest net472 0.00261ns 0.000948ns 0.00355ns 0 0 0 0 b
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 549μs 2.81μs 19.6μs 0 0 0 41.58 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 764μs 7.52μs 74.1μs 0 0 0 41.56 KB
master WriteAndFlushEnrichedTraces net472 932μs 6.09μs 60.6μs 8.33 0 0 53.42 KB
#6924 WriteAndFlushEnrichedTraces net6.0 625μs 7.93μs 77.7μs 0 0 0 41.62 KB
#6924 WriteAndFlushEnrichedTraces netcoreapp3.1 715μs 8.5μs 83.3μs 0 0 0 41.71 KB
#6924 WriteAndFlushEnrichedTraces net472 879μs 5.05μs 39.8μs 8.33 4.17 0 53.45 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net6.0 1.25μs 1.11ns 4.01ns 0.0126 0 0 1.03 KB
master ExecuteNonQuery netcoreapp3.1 1.89μs 3.45ns 12.9ns 0.00929 0 0 1.02 KB
master ExecuteNonQuery net472 2.08μs 4.88ns 18.9ns 0.155 0.0103 0 995 B
#6924 ExecuteNonQuery net6.0 1.27μs 2.56ns 9.58ns 0.0127 0 0 1.03 KB
#6924 ExecuteNonQuery netcoreapp3.1 1.73μs 4.05ns 15.2ns 0.00865 0 0 1.02 KB
#6924 ExecuteNonQuery net472 2.09μs 2.62ns 9.81ns 0.156 0.0104 0 995 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net6.0 1.33μs 1.52ns 5.88ns 0.0129 0 0 984 B
master CallElasticsearch netcoreapp3.1 1.54μs 1.63ns 5.86ns 0.00773 0 0 984 B
master CallElasticsearch net472 2.48μs 1.68ns 6.52ns 0.149 0 0 1 KB
master CallElasticsearchAsync net6.0 1.25μs 1.15ns 4.32ns 0.0125 0 0 960 B
master CallElasticsearchAsync netcoreapp3.1 1.65μs 2.22ns 8.31ns 0.00818 0 0 1.03 KB
master CallElasticsearchAsync net472 2.75μs 1.43ns 5.53ns 0.165 0 0 1.06 KB
#6924 CallElasticsearch net6.0 1.34μs 1ns 3.89ns 0.0134 0 0 984 B
#6924 CallElasticsearch netcoreapp3.1 1.5μs 0.937ns 3.51ns 0.00751 0 0 984 B
#6924 CallElasticsearch net472 2.52μs 1.41ns 5.45ns 0.152 0 0 1 KB
#6924 CallElasticsearchAsync net6.0 1.21μs 0.833ns 3.12ns 0.0119 0 0 960 B
#6924 CallElasticsearchAsync netcoreapp3.1 1.65μs 0.915ns 3.54ns 0.00831 0 0 1.03 KB
#6924 CallElasticsearchAsync net472 2.58μs 1.65ns 5.72ns 0.167 0 0 1.06 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net6.0 1.46μs 0.885ns 3.43ns 0.00712 0 0 960 B
master ExecuteAsync netcoreapp3.1 1.67μs 1.22ns 4.39ns 0.00833 0 0 960 B
master ExecuteAsync net472 1.79μs 0.767ns 2.87ns 0.143 0 0 923 B
#6924 ExecuteAsync net6.0 1.33μs 1.08ns 4.19ns 0.0134 0 0 960 B
#6924 ExecuteAsync netcoreapp3.1 1.69μs 2.21ns 8.28ns 0.00831 0 0 960 B
#6924 ExecuteAsync net472 1.86μs 0.647ns 2.33ns 0.14 0 0 923 B
Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net6.0 4.34μs 3.42ns 12.8ns 0.0221 0 0 2.32 KB
master SendAsync netcoreapp3.1 5.29μs 3.48ns 13.5ns 0.0263 0 0 2.86 KB
master SendAsync net472 7.51μs 6.94ns 26.9ns 0.487 0 0 3.13 KB
#6924 SendAsync net6.0 4.37μs 3.16ns 11.8ns 0.0219 0 0 2.32 KB
#6924 SendAsync netcoreapp3.1 5.37μs 9.1ns 31.5ns 0.0269 0 0 2.86 KB
#6924 SendAsync net472 7.39μs 3.66ns 13.2ns 0.482 0 0 3.13 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 1.54μs 7.35ns 31.2ns 0.0231 0 0 1.71 KB
master EnrichedLog netcoreapp3.1 2.28μs 3.53ns 12.7ns 0.0114 0 0 1.71 KB
master EnrichedLog net472 2.56μs 4.45ns 16.7ns 0.253 0 0 1.64 KB
#6924 EnrichedLog net6.0 1.57μs 1.69ns 6.55ns 0.0224 0 0 1.71 KB
#6924 EnrichedLog netcoreapp3.1 2.16μs 2.99ns 11.2ns 0.0213 0 0 1.71 KB
#6924 EnrichedLog net472 2.57μs 1.36ns 5.09ns 0.258 0 0 1.64 KB
Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 112μs 165ns 639ns 0 0 0 4.32 KB
master EnrichedLog netcoreapp3.1 115μs 185ns 717ns 0 0 0 4.32 KB
master EnrichedLog net472 149μs 266ns 1.03μs 0 0 0 4.51 KB
#6924 EnrichedLog net6.0 113μs 205ns 794ns 0 0 0 4.32 KB
#6924 EnrichedLog netcoreapp3.1 115μs 138ns 533ns 0 0 0 4.32 KB
#6924 EnrichedLog net472 151μs 290ns 1.12μs 0 0 0 4.51 KB
Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 3.17μs 2.22ns 8.32ns 0.0315 0 0 2.26 KB
master EnrichedLog netcoreapp3.1 4.16μs 1.93ns 7.21ns 0.0209 0 0 2.26 KB
master EnrichedLog net472 4.98μs 4.07ns 15.7ns 0.322 0 0 2.09 KB
#6924 EnrichedLog net6.0 3.21μs 2.11ns 7.9ns 0.0163 0 0 2.26 KB
#6924 EnrichedLog netcoreapp3.1 4.32μs 4.99ns 19.3ns 0.0216 0 0 2.26 KB
#6924 EnrichedLog net472 4.8μs 2.87ns 11.1ns 0.312 0 0 2.09 KB
Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net6.0 1.33μs 0.838ns 3.14ns 0.0133 0 0 1.15 KB
master SendReceive netcoreapp3.1 1.8μs 1.61ns 6.03ns 0.00901 0 0 1.15 KB
master SendReceive net472 2.01μs 2.61ns 10.1ns 0.18 0 0 1.16 KB
#6924 SendReceive net6.0 1.34μs 1.37ns 5.29ns 0.0134 0 0 1.15 KB
#6924 SendReceive netcoreapp3.1 1.75μs 1.49ns 5.56ns 0.00886 0 0 1.15 KB
#6924 SendReceive net472 2.06μs 0.94ns 3.52ns 0.175 0 0 1.16 KB
Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.75μs 1.72ns 6.68ns 0.0137 0 0 1.64 KB
master EnrichedLog netcoreapp3.1 4.03μs 4.29ns 16ns 0.0201 0 0 1.69 KB
master EnrichedLog net472 4.46μs 3.37ns 13ns 0.313 0 0 2.08 KB
#6924 EnrichedLog net6.0 2.76μs 3ns 11.6ns 0.0138 0 0 1.64 KB
#6924 EnrichedLog netcoreapp3.1 3.9μs 1.56ns 5.63ns 0.0196 0 0 1.69 KB
#6924 EnrichedLog net472 4.54μs 3.29ns 12.8ns 0.318 0 0 2.08 KB
Benchmarks.Trace.SpanBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net6.0 415ns 0.358ns 1.34ns 0.00832 0 0 584 B
master StartFinishSpan netcoreapp3.1 555ns 0.665ns 2.49ns 0.00555 0 0 584 B
master StartFinishSpan net472 654ns 0.393ns 1.42ns 0.0927 0 0 586 B
master StartFinishScope net6.0 503ns 0.393ns 1.42ns 0.0101 0 0 704 B
master StartFinishScope netcoreapp3.1 731ns 2.1ns 7.26ns 0.00733 0 0 704 B
master StartFinishScope net472 835ns 0.54ns 2.02ns 0.104 0 0 666 B
#6924 StartFinishSpan net6.0 424ns 0.228ns 0.885ns 0.00638 0 0 584 B
#6924 StartFinishSpan netcoreapp3.1 564ns 0.566ns 2.19ns 0.00563 0 0 584 B
#6924 StartFinishSpan net472 618ns 0.482ns 1.87ns 0.0923 0 0 586 B
#6924 StartFinishScope net6.0 480ns 0.306ns 1.15ns 0.00948 0 0 704 B
#6924 StartFinishScope netcoreapp3.1 804ns 0.729ns 2.73ns 0.00786 0 0 704 B
#6924 StartFinishScope net472 842ns 0.812ns 3.14ns 0.102 0 0 666 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net6.0 699ns 0.968ns 3.49ns 0.00702 0 0 704 B
master RunOnMethodBegin netcoreapp3.1 895ns 1.12ns 4.2ns 0.00897 0 0 704 B
master RunOnMethodBegin net472 1.04μs 0.328ns 1.23ns 0.104 0 0 666 B
#6924 RunOnMethodBegin net6.0 655ns 0.66ns 2.56ns 0.01 0 0 704 B
#6924 RunOnMethodBegin netcoreapp3.1 938ns 1.01ns 3.9ns 0.00933 0 0 704 B
#6924 RunOnMethodBegin net472 1.08μs 0.284ns 1.02ns 0.103 0 0 666 B

@andrewlock
Copy link
Member

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

  • Welch test with statistical test for significance of 5%
  • Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6924) - mean (70ms)  : 66, 74
     .   : milestone, 70,
    master - mean (70ms)  : 66, 73
     .   : milestone, 70,

    section CallTarget+Inlining+NGEN
    This PR (6924) - mean (1,012ms)  : 989, 1036
     .   : milestone, 1012,
    master - mean (1,013ms)  : 987, 1038
     .   : milestone, 1013,
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6924) - mean (103ms)  : 101, 105
     .   : milestone, 103,
    master - mean (103ms)  : 100, 106
     .   : milestone, 103,

    section CallTarget+Inlining+NGEN
    This PR (6924) - mean (693ms)  : 677, 708
     .   : milestone, 693,
    master - mean (692ms)  : 669, 715
     .   : milestone, 692,
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6924) - mean (90ms)  : 87, 92
     .   : milestone, 90,
    master - mean (89ms)  : 87, 92
     .   : milestone, 89,

    section CallTarget+Inlining+NGEN
    This PR (6924) - mean (653ms)  : 631, 674
     .   : milestone, 653,
    master - mean (655ms)  : 634, 675
     .   : milestone, 655,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6924) - mean (191ms)  : 187, 194
     .   : milestone, 191,
    master - mean (190ms)  : 186, 194
     .   : milestone, 190,

    section CallTarget+Inlining+NGEN
    This PR (6924) - mean (1,111ms)  : 1086, 1137
     .   : milestone, 1111,
    master - mean (1,109ms)  : 1077, 1142
     .   : milestone, 1109,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6924) - mean (269ms)  : 265, 273
     .   : milestone, 269,
    master - mean (269ms)  : 265, 273
     .   : milestone, 269,

    section CallTarget+Inlining+NGEN
    This PR (6924) - mean (880ms)  : 847, 912
     .   : milestone, 880,
    master - mean (879ms)  : 847, 911
     .   : milestone, 879,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6924) - mean (262ms)  : 258, 266
     .   : milestone, 262,
    master - mean (261ms)  : 257, 265
     .   : milestone, 261,

    section CallTarget+Inlining+NGEN
    This PR (6924) - mean (865ms)  : 831, 899
     .   : milestone, 865,
    master - mean (874ms)  : 845, 903
     .   : milestone, 874,
Loading

andrewlock
andrewlock approved these changes May 6, 2025
View reviewed changes
Copy link
Member

@andrewlock andrewlock left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Taking your word for it, thanks! 🙈

@faydef faydef merged commit 225488b into master May 6, 2025
128 checks passed
@faydef faydef deleted the fayssal/update-bp-infra branch May 6, 2025 15:15
@github-actions github-actions bot added this to the vNext-v3 milestone May 6, 2025
@andrewlock andrewlock added area:builds project files, build scripts, pipelines, versioning, releases, packages area:benchmarks Benchmarks, throughput tests, Crank, Bombardier, etc labels May 23, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@andrewlock andrewlock andrewlock approved these changes

Assignees
No one assigned
Labels
area:benchmarks Benchmarks, throughput tests, Crank, Bombardier, etc area:builds project files, build scripts, pipelines, versioning, releases, packages
Projects
None yet
Milestone
3.17.0
Development

Successfully merging this pull request may close these issues.
2 participants
@faydef @andrewlock