Skip to content

Mark a couple of tests as flaky #7026

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
May 29, 2025
Merged

Mark a couple of tests as flaky #7026

merged 2 commits into from
May 29, 2025

Conversation

andrewlock
Copy link
Member

Summary of changes

Marks a couple of tests as flaky

Reason for change

They're flaky.

  • The discovery service one is inherently flaky, as it's time-based
  • The HttpMessageHandler is notoriously flaky. It throws a crashing ObjectDisposedException from the HttpListener. This has been about for ages, and we never figured it out, so just marking it

Implementation details

Mark them as flaky

Test coverage

Indeed

@andrewlock andrewlock added the area:tests unit tests, integration tests label May 28, 2025
@andrewlock andrewlock requested review from a team as code owners May 28, 2025 18:11
@datadog-datadog-prod-us1
Copy link

datadog-datadog-prod-us1 bot commented May 28, 2025 •
edited
Loading

Datadog Report

All test runs 558c521 🔗

✅ 2 Total Test Services: 0 Failed, 2 Passed

Test Services
Service Name Failed Known Flaky New Flaky Passed Skipped Total Time Test Service View
dd-trace-dotnet 0 0 0 246469 2287 15h 2m 12.3s Link
exploration_tests 0 0 0 22085 3 2m 12.81s Link

@andrewlock
Copy link
Member Author

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

  • Welch test with statistical test for significance of 5%
  • Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.8) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (7026) - mean (68ms)  : 65, 71
     .   : milestone, 68,
    master - mean (68ms)  : 65, 70
     .   : milestone, 68,

    section CallTarget+Inlining+NGEN
    This PR (7026) - mean (1,005ms)  : 985, 1025
     .   : milestone, 1005,
    master - mean (1,007ms)  : 980, 1034
     .   : milestone, 1007,
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (7026) - mean (102ms)  : 100, 103
     .   : milestone, 102,
    master - mean (102ms)  : 100, 104
     .   : milestone, 102,

    section CallTarget+Inlining+NGEN
    This PR (7026) - mean (691ms)  : 672, 711
     .   : milestone, 691,
    master - mean (695ms)  : 675, 714
     .   : milestone, 695,
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (7026) - mean (89ms)  : 87, 91
     .   : milestone, 89,
    master - mean (89ms)  : 87, 92
     .   : milestone, 89,

    section CallTarget+Inlining+NGEN
    This PR (7026) - mean (650ms)  : 628, 672
     .   : milestone, 650,
    master - mean (656ms)  : 633, 679
     .   : milestone, 656,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.8) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (7026) - mean (191ms)  : 183, 198
     .   : milestone, 191,
    master - mean (189ms)  : 186, 192
     .   : milestone, 189,

    section CallTarget+Inlining+NGEN
    This PR (7026) - mean (1,108ms)  : 1080, 1137
     .   : milestone, 1108,
    master - mean (1,111ms)  : 1091, 1131
     .   : milestone, 1111,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (7026) - mean (268ms)  : 264, 272
     .   : milestone, 268,
    master - mean (269ms)  : 265, 273
     .   : milestone, 269,

    section CallTarget+Inlining+NGEN
    This PR (7026) - mean (875ms)  : 844, 906
     .   : milestone, 875,
    master - mean (878ms)  : 845, 912
     .   : milestone, 878,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (7026) - mean (261ms)  : 258, 264
     .   : milestone, 261,
    master - mean (261ms)  : 258, 265
     .   : milestone, 261,

    section CallTarget+Inlining+NGEN
    This PR (7026) - mean (867ms)  : 829, 905
     .   : milestone, 867,
    master - mean (871ms)  : 841, 900
     .   : milestone, 871,
Loading

@pr-commenter
Copy link

pr-commenter bot commented May 28, 2025

Benchmarks

Benchmarks Report for benchmark platform 🐌

Benchmarks for #99 compared to master:

  • All benchmarks have the same speed
  • 3 benchmarks have fewer allocations
  • 7 benchmarks have more allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #99

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.ActivityBenchmark.StartStopWithChild‑netcoreapp3.1 5.71 KB 5.75 KB 34 B 0.60%
Benchmarks.Trace.ActivityBenchmark.StartStopWithChild‑net6.0 5.54 KB 5.57 KB 31 B 0.56%

Fewer allocations 🎉 in #99

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.ActivityBenchmark.StartStopWithChild‑net472 6.21 KB 6.16 KB -49 B -0.79%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartStopWithChild net6.0 10.7μs 58.7ns 342ns 0 0 0 5.54 KB
master StartStopWithChild netcoreapp3.1 13.9μs 62.6ns 242ns 0 0 0 5.71 KB
master StartStopWithChild net472 22.4μs 117ns 549ns 0.909 0.303 0 6.21 KB
#99 StartStopWithChild net6.0 10.6μs 51.4ns 199ns 0 0 0 5.57 KB
#99 StartStopWithChild netcoreapp3.1 13.3μs 68.4ns 306ns 0 0 0 5.75 KB
#99 StartStopWithChild net472 21.5μs 112ns 582ns 0.994 0.331 0.11 6.16 KB
Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #99

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.AgentWriterBenchmark.WriteAndFlushEnrichedTraces‑net472 3.31 KB 3.35 KB 46 B 1.39%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 923μs 40.6ns 152ns 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 1.02ms 56.6ns 196ns 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces net472 1.22ms 931ns 3.6μs 0 0 0 3.31 KB
#99 WriteAndFlushEnrichedTraces net6.0 911μs 40.2ns 150ns 0 0 0 2.71 KB
#99 WriteAndFlushEnrichedTraces netcoreapp3.1 1.03ms 205ns 795ns 0 0 0 2.7 KB
#99 WriteAndFlushEnrichedTraces net472 1.23ms 32.7ns 113ns 0 0 0 3.35 KB
Benchmarks.Trace.Asm.AppSecBodyBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net6.0 329μs 254ns 982ns 0 0 0 197.06 KB
master AllCycleSimpleBody netcoreapp3.1 494μs 722ns 2.8μs 0 0 0 204.77 KB
master AllCycleSimpleBody net472 438μs 90.9ns 340ns 36.6 2.16 0 236.35 KB
master AllCycleMoreComplexBody net6.0 342μs 243ns 943ns 0 0 0 200.56 KB
master AllCycleMoreComplexBody netcoreapp3.1 489μs 340ns 1.27μs 0 0 0 208.18 KB
master AllCycleMoreComplexBody net472 448μs 134ns 500ns 37.9 2.23 0 239.88 KB
master ObjectExtractorSimpleBody net6.0 329ns 0.0837ns 0.302ns 0 0 0 280 B
master ObjectExtractorSimpleBody netcoreapp3.1 402ns 0.296ns 1.15ns 0 0 0 272 B
master ObjectExtractorSimpleBody net472 302ns 0.0178ns 0.0689ns 0.0441 0 0 281 B
master ObjectExtractorMoreComplexBody net6.0 6.39μs 7.82ns 30.3ns 0 0 0 3.78 KB
master ObjectExtractorMoreComplexBody netcoreapp3.1 7.66μs 35.2ns 141ns 0 0 0 3.69 KB
master ObjectExtractorMoreComplexBody net472 6.7μs 2.65ns 10.3ns 0.603 0 0 3.8 KB
#99 AllCycleSimpleBody net6.0 326μs 1.48μs 5.93μs 0 0 0 197.06 KB
#99 AllCycleSimpleBody netcoreapp3.1 501μs 1.56μs 6.06μs 0 0 0 204.77 KB
#99 AllCycleSimpleBody net472 437μs 96.2ns 360ns 36.6 2.16 0 236.35 KB
#99 AllCycleMoreComplexBody net6.0 342μs 1.7μs 7.77μs 0 0 0 200.68 KB
#99 AllCycleMoreComplexBody netcoreapp3.1 490μs 1.98μs 7.15μs 0 0 0 208.18 KB
#99 AllCycleMoreComplexBody net472 447μs 117ns 438ns 36.6 2.16 0 239.87 KB
#99 ObjectExtractorSimpleBody net6.0 314ns 0.716ns 2.77ns 0 0 0 280 B
#99 ObjectExtractorSimpleBody netcoreapp3.1 397ns 1.92ns 7.45ns 0 0 0 272 B
#99 ObjectExtractorSimpleBody net472 305ns 0.0385ns 0.149ns 0.0441 0 0 281 B
#99 ObjectExtractorMoreComplexBody net6.0 6.32μs 29.3ns 113ns 0 0 0 3.78 KB
#99 ObjectExtractorMoreComplexBody netcoreapp3.1 7.75μs 35.7ns 138ns 0 0 0 3.69 KB
#99 ObjectExtractorMoreComplexBody net472 6.69μs 3.92ns 15.2ns 0.602 0 0 3.8 KB
Benchmarks.Trace.Asm.AppSecEncoderBenchmark - Unknown 🤷 Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EncodeArgs net6.0 N/A N/A N/A NaN NaN NaN 0 b
master EncodeArgs netcoreapp3.1 N/A N/A N/A NaN NaN NaN 0 b
master EncodeArgs net472 N/A N/A N/A NaN NaN NaN 0 b
master EncodeLegacyArgs net6.0 N/A N/A N/A NaN NaN NaN 0 b
master EncodeLegacyArgs netcoreapp3.1 N/A N/A N/A NaN NaN NaN 0 b
master EncodeLegacyArgs net472 N/A N/A N/A NaN NaN NaN 0 b
#99 EncodeArgs net6.0 N/A N/A N/A NaN NaN NaN 0 b
#99 EncodeArgs netcoreapp3.1 N/A N/A N/A NaN NaN NaN 0 b
#99 EncodeArgs net472 N/A N/A N/A NaN NaN NaN 0 b
#99 EncodeLegacyArgs net6.0 N/A N/A N/A NaN NaN NaN 0 b
#99 EncodeLegacyArgs netcoreapp3.1 N/A N/A N/A NaN NaN NaN 0 b
#99 EncodeLegacyArgs net472 N/A N/A N/A NaN NaN NaN 0 b
Benchmarks.Trace.Asm.AppSecWafBenchmark - Unknown 🤷 Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunWafRealisticBenchmark net6.0 N/A N/A N/A NaN NaN NaN 0 b
master RunWafRealisticBenchmark netcoreapp3.1 N/A N/A N/A NaN NaN NaN 0 b
master RunWafRealisticBenchmark net472 N/A N/A N/A NaN NaN NaN 0 b
master RunWafRealisticBenchmarkWithAttack net6.0 N/A N/A N/A NaN NaN NaN 0 b
master RunWafRealisticBenchmarkWithAttack netcoreapp3.1 N/A N/A N/A NaN NaN NaN 0 b
master RunWafRealisticBenchmarkWithAttack net472 N/A N/A N/A NaN NaN NaN 0 b
#99 RunWafRealisticBenchmark net6.0 N/A N/A N/A NaN NaN NaN 0 b
#99 RunWafRealisticBenchmark netcoreapp3.1 N/A N/A N/A NaN NaN NaN 0 b
#99 RunWafRealisticBenchmark net472 N/A N/A N/A NaN NaN NaN 0 b
#99 RunWafRealisticBenchmarkWithAttack net6.0 N/A N/A N/A NaN NaN NaN 0 b
#99 RunWafRealisticBenchmarkWithAttack netcoreapp3.1 N/A N/A N/A NaN NaN NaN 0 b
#99 RunWafRealisticBenchmarkWithAttack net472 N/A N/A N/A NaN NaN NaN 0 b
Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net6.0 61.5μs 93.3ns 361ns 0 0 0 14.53 KB
master SendRequest netcoreapp3.1 70.4μs 191ns 714ns 0 0 0 17.42 KB
master SendRequest net472 0.00653ns 0.0017ns 0.00657ns 0 0 0 0 b
#99 SendRequest net6.0 60.9μs 49.4ns 171ns 0 0 0 14.53 KB
#99 SendRequest netcoreapp3.1 72.4μs 107ns 415ns 0 0 0 17.42 KB
#99 SendRequest net472 0.00831ns 0.00214ns 0.00828ns 0 0 0 0 b
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #99

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑netcoreapp3.1 41.93 KB 42.2 KB 273 B 0.65%

Fewer allocations 🎉 in #99

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑net6.0 41.89 KB 41.59 KB -305 B -0.73%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 722μs 378ns 1.46μs 0 0 0 41.89 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 639μs 3.53μs 26μs 0 0 0 41.93 KB
master WriteAndFlushEnrichedTraces net472 968μs 5.17μs 28.8μs 8.33 0 0 56.08 KB
#99 WriteAndFlushEnrichedTraces net6.0 713μs 987ns 3.82μs 0 0 0 41.59 KB
#99 WriteAndFlushEnrichedTraces netcoreapp3.1 672μs 572ns 2.06μs 0 0 0 42.2 KB
#99 WriteAndFlushEnrichedTraces net472 908μs 2.34μs 9.05μs 4.46 0 0 55.97 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net6.0 1.94μs 6.03ns 23.4ns 0 0 0 1.03 KB
master ExecuteNonQuery netcoreapp3.1 2.46μs 9.67ns 37.4ns 0 0 0 1.02 KB
master ExecuteNonQuery net472 2.71μs 3.38ns 13.1ns 0.149 0.0135 0 995 B
#99 ExecuteNonQuery net6.0 1.89μs 7.96ns 29.8ns 0 0 0 1.03 KB
#99 ExecuteNonQuery netcoreapp3.1 2.56μs 4.68ns 18.1ns 0 0 0 1.02 KB
#99 ExecuteNonQuery net472 2.69μs 2.87ns 10.3ns 0.15 0.0137 0 995 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net6.0 1.77μs 7.85ns 30.4ns 0 0 0 1.04 KB
master CallElasticsearch netcoreapp3.1 2.29μs 4.16ns 16.1ns 0 0 0 1.04 KB
master CallElasticsearch net472 3.42μs 1.68ns 6.5ns 0.154 0 0 1.05 KB
master CallElasticsearchAsync net6.0 1.85μs 9.19ns 39ns 0 0 0 1.02 KB
master CallElasticsearchAsync netcoreapp3.1 2.43μs 11.1ns 43.1ns 0 0 0 1.09 KB
master CallElasticsearchAsync net472 3.6μs 4.48ns 17.4ns 0.163 0 0 1.11 KB
#99 CallElasticsearch net6.0 1.77μs 7.06ns 27.3ns 0 0 0 1.04 KB
#99 CallElasticsearch netcoreapp3.1 2.34μs 10.6ns 40.9ns 0 0 0 1.04 KB
#99 CallElasticsearch net472 3.55μs 4.04ns 15.7ns 0.159 0 0 1.05 KB
#99 CallElasticsearchAsync net6.0 1.84μs 8.79ns 36.2ns 0 0 0 1.02 KB
#99 CallElasticsearchAsync netcoreapp3.1 2.42μs 10.1ns 39.2ns 0 0 0 1.09 KB
#99 CallElasticsearchAsync net472 3.76μs 4.23ns 16.4ns 0.169 0 0 1.11 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net6.0 1.86μs 2.82ns 10.6ns 0 0 0 960 B
master ExecuteAsync netcoreapp3.1 2.26μs 5.07ns 19.6ns 0 0 0 960 B
master ExecuteAsync net472 2.58μs 2.76ns 10.7ns 0.142 0 0 923 B
#99 ExecuteAsync net6.0 1.8μs 8.95ns 36.9ns 0 0 0 960 B
#99 ExecuteAsync netcoreapp3.1 2.3μs 11.2ns 44.8ns 0 0 0 960 B
#99 ExecuteAsync net472 2.63μs 2.85ns 11ns 0.144 0 0 923 B
Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net6.0 6.85μs 6.24ns 24.2ns 0 0 0 2.37 KB
master SendAsync netcoreapp3.1 8.38μs 19.9ns 77.1ns 0 0 0 2.9 KB
master SendAsync net472 12.6μs 3.83ns 13.3ns 0.504 0 0 3.19 KB
#99 SendAsync net6.0 6.65μs 16.4ns 63.7ns 0 0 0 2.37 KB
#99 SendAsync netcoreapp3.1 8.62μs 10.1ns 39.3ns 0 0 0 2.9 KB
#99 SendAsync net472 12.1μs 10.1ns 39.1ns 0.482 0 0 3.19 KB
Benchmarks.Trace.Iast.StringAspectsBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #99

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑net472 57.34 KB 65.54 KB 8.19 KB 14.29%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑netcoreapp3.1 258.74 KB 275.3 KB 16.57 KB 6.40%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑net6.0 43.36 KB 43.82 KB 464 B 1.07%

Fewer allocations 🎉 in #99

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑net472 294.38 KB 289.66 KB -4.72 KB -1.60%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StringConcatBenchmark net6.0 41.4μs 207ns 879ns 0 0 0 43.36 KB
master StringConcatBenchmark netcoreapp3.1 57.5μs 787ns 7.87μs 0 0 0 42.84 KB
master StringConcatBenchmark net472 56.9μs 205ns 767ns 0 0 0 57.34 KB
master StringConcatAspectBenchmark net6.0 506μs 2.48μs 10.5μs 0 0 0 276.35 KB
master StringConcatAspectBenchmark netcoreapp3.1 521μs 2.62μs 11.1μs 0 0 0 258.74 KB
master StringConcatAspectBenchmark net472 406μs 1.91μs 7.86μs 0 0 0 294.38 KB
#99 StringConcatBenchmark net6.0 45.3μs 232ns 1.2μs 0 0 0 43.82 KB
#99 StringConcatBenchmark netcoreapp3.1 50.7μs 315ns 2.99μs 0 0 0 42.8 KB
#99 StringConcatBenchmark net472 58μs 261ns 975ns 0 0 0 65.54 KB
#99 StringConcatAspectBenchmark net6.0 482μs 1.36μs 4.72μs 0 0 0 276.83 KB
#99 StringConcatAspectBenchmark netcoreapp3.1 529μs 2.41μs 9.33μs 0 0 0 275.3 KB
#99 StringConcatAspectBenchmark net472 408μs 1.97μs 8.1μs 0 0 0 289.66 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.58μs 12.3ns 50.6ns 0 0 0 1.76 KB
master EnrichedLog netcoreapp3.1 3.51μs 6.57ns 25.4ns 0 0 0 1.76 KB
master EnrichedLog net472 3.93μs 4.7ns 18.2ns 0.255 0 0 1.69 KB
#99 EnrichedLog net6.0 2.58μs 13.7ns 67ns 0 0 0 1.76 KB
#99 EnrichedLog netcoreapp3.1 3.38μs 2.15ns 8.06ns 0 0 0 1.76 KB
#99 EnrichedLog net472 3.91μs 4.13ns 16ns 0.255 0 0 1.69 KB
Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 123μs 38.2ns 143ns 0 0 0 4.37 KB
master EnrichedLog netcoreapp3.1 129μs 115ns 431ns 0 0 0 4.37 KB
master EnrichedLog net472 168μs 61.5ns 238ns 0 0 0 4.57 KB
#99 EnrichedLog net6.0 122μs 61.9ns 223ns 0 0 0 4.37 KB
#99 EnrichedLog netcoreapp3.1 127μs 103ns 371ns 0 0 0 4.37 KB
#99 EnrichedLog net472 166μs 84.7ns 317ns 0 0 0 4.57 KB
Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 5.16μs 3.53ns 12.2ns 0 0 0 2.32 KB
master EnrichedLog netcoreapp3.1 6.74μs 21.3ns 82.6ns 0 0 0 2.32 KB
master EnrichedLog net472 7.36μs 16.1ns 62.3ns 0.328 0 0 2.14 KB
#99 EnrichedLog net6.0 4.97μs 5.79ns 21.7ns 0 0 0 2.32 KB
#99 EnrichedLog netcoreapp3.1 6.64μs 23.5ns 88.1ns 0 0 0 2.32 KB
#99 EnrichedLog net472 7.26μs 5.42ns 21ns 0.324 0 0 2.14 KB
Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net6.0 2μs 10.7ns 59.8ns 0 0 0 1.21 KB
master SendReceive netcoreapp3.1 2.53μs 10.3ns 40ns 0 0 0 1.21 KB
master SendReceive net472 3.01μs 0.997ns 3.86ns 0.18 0 0 1.21 KB
#99 SendReceive net6.0 1.96μs 2.77ns 10.7ns 0 0 0 1.21 KB
#99 SendReceive netcoreapp3.1 2.74μs 8.18ns 31.7ns 0 0 0 1.21 KB
#99 SendReceive net472 3μs 2.36ns 8.83ns 0.18 0 0 1.21 KB
Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 4.04μs 13.5ns 52.4ns 0 0 0 1.64 KB
master EnrichedLog netcoreapp3.1 5.53μs 15.3ns 57.3ns 0 0 0 1.69 KB
master EnrichedLog net472 6.71μs 4.99ns 19.3ns 0.298 0 0 2.08 KB
#99 EnrichedLog net6.0 4.18μs 7.84ns 30.4ns 0 0 0 1.64 KB
#99 EnrichedLog netcoreapp3.1 5.52μs 15.8ns 61.4ns 0 0 0 1.69 KB
#99 EnrichedLog net472 6.84μs 7.76ns 30ns 0.308 0 0 2.08 KB
Benchmarks.Trace.SpanBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net6.0 741ns 3.89ns 19.1ns 0 0 0 584 B
master StartFinishSpan netcoreapp3.1 942ns 4.82ns 21ns 0 0 0 584 B
master StartFinishSpan net472 931ns 0.0956ns 0.358ns 0.0899 0 0 586 B
master StartFinishScope net6.0 914ns 4.47ns 17.9ns 0 0 0 704 B
master StartFinishScope netcoreapp3.1 1.15μs 5.62ns 23.2ns 0 0 0 704 B
master StartFinishScope net472 1.12μs 2.14ns 8.3ns 0.102 0 0 666 B
#99 StartFinishSpan net6.0 748ns 0.189ns 0.733ns 0 0 0 584 B
#99 StartFinishSpan netcoreapp3.1 944ns 0.4ns 1.5ns 0 0 0 584 B
#99 StartFinishSpan net472 898ns 0.206ns 0.797ns 0.09 0 0 586 B
#99 StartFinishScope net6.0 879ns 4.47ns 21.9ns 0 0 0 704 B
#99 StartFinishScope netcoreapp3.1 1.12μs 5.97ns 29.9ns 0 0 0 704 B
#99 StartFinishScope net472 1.08μs 0.0821ns 0.318ns 0.103 0 0 666 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net6.0 1.04μs 1.01ns 3.93ns 0 0 0 704 B
master RunOnMethodBegin netcoreapp3.1 1.35μs 0.804ns 3.01ns 0 0 0 704 B
master RunOnMethodBegin net472 1.42μs 0.705ns 2.64ns 0.0999 0 0 666 B
#99 RunOnMethodBegin net6.0 1.04μs 0.369ns 1.38ns 0 0 0 704 B
#99 RunOnMethodBegin netcoreapp3.1 1.34μs 6.22ns 24.9ns 0 0 0 704 B
#99 RunOnMethodBegin net472 1.35μs 0.654ns 2.53ns 0.101 0 0 666 B

@andrewlock andrewlock merged commit 6fdbc84 into master May 29, 2025
126 of 129 checks passed
@andrewlock andrewlock deleted the andrew/ci/mark-flaky-tests branch May 29, 2025 08:58
@github-actions github-actions bot added this to the vNext-v3 milestone May 29, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@bouwkast bouwkast bouwkast approved these changes

@NachoEchevarria NachoEchevarria NachoEchevarria approved these changes

Assignees
No one assigned
Labels
area:tests unit tests, integration tests
Projects
None yet
Milestone
3.18.0
Development

Successfully merging this pull request may close these issues.
3 participants
@andrewlock @bouwkast @NachoEchevarria