Skip to content

Skip LineProbeUnboundProbeBecomesBoundTest because is too flaky #7000

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
May 23, 2025
Merged

Skip LineProbeUnboundProbeBecomesBoundTest because is too flaky #7000

merged 1 commit into from
May 23, 2025

Conversation

dudikeleti
Copy link
Contributor

Summary of changes

Skip LineProbeUnboundProbeBecomesBoundTest because is too flaky: 'Log file was not found for path'

@dudikeleti dudikeleti requested a review from a team as a code owner May 23, 2025 09:08
@dudikeleti dudikeleti requested a review from andrewlock May 23, 2025 09:08
@github-actions github-actions bot added the area:tests unit tests, integration tests label May 23, 2025
andrewlock
andrewlock approved these changes May 23, 2025
View reviewed changes
Copy link
Member

@andrewlock andrewlock left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@datadog-datadog-prod-us1
Copy link

datadog-datadog-prod-us1 bot commented May 23, 2025 •
edited
Loading

Datadog Report

All test runs 598a399 🔗

✅ 2 Total Test Services: 0 Failed, 2 Passed
⌛ 1 with Regressions

Test Services
Service Name Failed Known Flaky New Flaky Passed Skipped Total Time Performance Regressions Test Service View
dd-trace-dotnet 0 0 0 263102 6479 39h 37m 3.92s 1 Link
exploration_tests 0 0 0 7433 0 10m 35.81s 0 Link

⌛ Performance Regressions vs Default Branch (1)

  • Profiler_cpu_walltime - scenarios 3.54s (+198.36ms, +6%) - Details

@pr-commenter
Copy link

pr-commenter bot commented May 23, 2025

Benchmarks

Benchmarks Report for benchmark platform 🐌

Benchmarks for #7000 compared to master:

  • All benchmarks have the same speed
  • 3 benchmarks have fewer allocations
  • 4 benchmarks have more allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ Fewer allocations 🎉

Fewer allocations 🎉 in #7000

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.ActivityBenchmark.StartStopWithChild‑netcoreapp3.1 5.74 KB 5.61 KB -133 B -2.32%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartStopWithChild net6.0 10.2μs 58.1ns 419ns 0 0 0 5.56 KB
master StartStopWithChild netcoreapp3.1 13.2μs 68.5ns 328ns 0 0 0 5.74 KB
master StartStopWithChild net472 21.6μs 101ns 392ns 1 0.334 0.111 6.15 KB
#7000 StartStopWithChild net6.0 10.3μs 56.4ns 357ns 0 0 0 5.54 KB
#7000 StartStopWithChild netcoreapp3.1 13.1μs 58ns 217ns 0 0 0 5.61 KB
#7000 StartStopWithChild net472 22.2μs 119ns 621ns 0.958 0.213 0 6.16 KB
Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 931μs 64.7ns 242ns 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 1.02ms 253ns 947ns 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces net472 1.21ms 315ns 1.22μs 0 0 0 3.35 KB
#7000 WriteAndFlushEnrichedTraces net6.0 940μs 279ns 1.04μs 0 0 0 2.7 KB
#7000 WriteAndFlushEnrichedTraces netcoreapp3.1 1.06ms 63.1ns 227ns 0 0 0 2.7 KB
#7000 WriteAndFlushEnrichedTraces net472 1.23ms 796ns 3.08μs 0 0 0 3.35 KB
Benchmarks.Trace.Asm.AppSecBodyBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net6.0 324μs 1.64μs 7.84μs 0 0 0 197.06 KB
master AllCycleSimpleBody netcoreapp3.1 471μs 1.34μs 5.2μs 0 0 0 204.77 KB
master AllCycleSimpleBody net472 436μs 131ns 491ns 36.6 2.16 0 236.35 KB
master AllCycleMoreComplexBody net6.0 338μs 1.25μs 4.86μs 0 0 0 200.56 KB
master AllCycleMoreComplexBody netcoreapp3.1 497μs 1.5μs 5.82μs 0 0 0 208.18 KB
master AllCycleMoreComplexBody net472 444μs 108ns 404ns 36.6 2.16 0 239.87 KB
master ObjectExtractorSimpleBody net6.0 312ns 1.52ns 6.27ns 0 0 0 280 B
master ObjectExtractorSimpleBody netcoreapp3.1 398ns 1.95ns 8.25ns 0 0 0 272 B
master ObjectExtractorSimpleBody net472 301ns 0.0411ns 0.154ns 0.0434 0 0 281 B
master ObjectExtractorMoreComplexBody net6.0 6.42μs 32ns 136ns 0 0 0 3.78 KB
master ObjectExtractorMoreComplexBody netcoreapp3.1 7.69μs 38.2ns 166ns 0 0 0 3.69 KB
master ObjectExtractorMoreComplexBody net472 6.71μs 2.45ns 9.5ns 0.57 0 0 3.8 KB
#7000 AllCycleSimpleBody net6.0 331μs 525ns 1.96μs 0 0 0 197.06 KB
#7000 AllCycleSimpleBody netcoreapp3.1 502μs 1.33μs 4.96μs 0 0 0 204.77 KB
#7000 AllCycleSimpleBody net472 439μs 118ns 456ns 36.6 2.16 0 236.35 KB
#7000 AllCycleMoreComplexBody net6.0 341μs 289ns 1.04μs 0 0 0 200.56 KB
#7000 AllCycleMoreComplexBody netcoreapp3.1 495μs 456ns 1.77μs 0 0 0 208.18 KB
#7000 AllCycleMoreComplexBody net472 449μs 564ns 2.18μs 37.9 2.23 0 239.88 KB
#7000 ObjectExtractorSimpleBody net6.0 318ns 1.57ns 6.46ns 0 0 0 280 B
#7000 ObjectExtractorSimpleBody netcoreapp3.1 397ns 2.09ns 10.7ns 0 0 0 272 B
#7000 ObjectExtractorSimpleBody net472 303ns 0.122ns 0.471ns 0.0443 0 0 281 B
#7000 ObjectExtractorMoreComplexBody net6.0 6.49μs 20.6ns 79.7ns 0 0 0 3.78 KB
#7000 ObjectExtractorMoreComplexBody netcoreapp3.1 7.82μs 28.6ns 111ns 0 0 0 3.69 KB
#7000 ObjectExtractorMoreComplexBody net472 6.68μs 3.15ns 11.8ns 0.602 0 0 3.8 KB
Benchmarks.Trace.Asm.AppSecEncoderBenchmark - Unknown 🤷 Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EncodeArgs net6.0 N/A N/A N/A NaN NaN NaN 0 b
master EncodeArgs netcoreapp3.1 N/A N/A N/A NaN NaN NaN 0 b
master EncodeArgs net472 N/A N/A N/A NaN NaN NaN 0 b
master EncodeLegacyArgs net6.0 N/A N/A N/A NaN NaN NaN 0 b
master EncodeLegacyArgs netcoreapp3.1 N/A N/A N/A NaN NaN NaN 0 b
master EncodeLegacyArgs net472 N/A N/A N/A NaN NaN NaN 0 b
#7000 EncodeArgs net6.0 N/A N/A N/A NaN NaN NaN 0 b
#7000 EncodeArgs netcoreapp3.1 N/A N/A N/A NaN NaN NaN 0 b
#7000 EncodeArgs net472 N/A N/A N/A NaN NaN NaN 0 b
#7000 EncodeLegacyArgs net6.0 N/A N/A N/A NaN NaN NaN 0 b
#7000 EncodeLegacyArgs netcoreapp3.1 N/A N/A N/A NaN NaN NaN 0 b
#7000 EncodeLegacyArgs net472 N/A N/A N/A NaN NaN NaN 0 b
Benchmarks.Trace.Asm.AppSecWafBenchmark - Unknown 🤷 Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunWafRealisticBenchmark net6.0 N/A N/A N/A NaN NaN NaN 0 b
master RunWafRealisticBenchmark netcoreapp3.1 N/A N/A N/A NaN NaN NaN 0 b
master RunWafRealisticBenchmark net472 N/A N/A N/A NaN NaN NaN 0 b
master RunWafRealisticBenchmarkWithAttack net6.0 N/A N/A N/A NaN NaN NaN 0 b
master RunWafRealisticBenchmarkWithAttack netcoreapp3.1 N/A N/A N/A NaN NaN NaN 0 b
master RunWafRealisticBenchmarkWithAttack net472 N/A N/A N/A NaN NaN NaN 0 b
#7000 RunWafRealisticBenchmark net6.0 N/A N/A N/A NaN NaN NaN 0 b
#7000 RunWafRealisticBenchmark netcoreapp3.1 N/A N/A N/A NaN NaN NaN 0 b
#7000 RunWafRealisticBenchmark net472 N/A N/A N/A NaN NaN NaN 0 b
#7000 RunWafRealisticBenchmarkWithAttack net6.0 N/A N/A N/A NaN NaN NaN 0 b
#7000 RunWafRealisticBenchmarkWithAttack netcoreapp3.1 N/A N/A N/A NaN NaN NaN 0 b
#7000 RunWafRealisticBenchmarkWithAttack net472 N/A N/A N/A NaN NaN NaN 0 b
Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net6.0 60.8μs 34.4ns 133ns 0 0 0 14.53 KB
master SendRequest netcoreapp3.1 72.1μs 224ns 866ns 0 0 0 17.42 KB
master SendRequest net472 0.0118ns 0.00277ns 0.0107ns 0 0 0 0 b
#7000 SendRequest net6.0 59.7μs 53.9ns 209ns 0 0 0 14.53 KB
#7000 SendRequest netcoreapp3.1 72.9μs 132ns 511ns 0 0 0 17.42 KB
#7000 SendRequest net472 0.0113ns 0.00137ns 0.0053ns 0 0 0 0 b
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #7000

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑net6.0 41.8 KB 42.36 KB 560 B 1.34%

Fewer allocations 🎉 in #7000

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑net472 56.56 KB 55.59 KB -970 B -1.71%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 706μs 3.77μs 19.6μs 0 0 0 41.8 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 707μs 3.93μs 23.9μs 0 0 0 42.02 KB
master WriteAndFlushEnrichedTraces net472 936μs 2.99μs 11.2μs 8.33 0 0 56.56 KB
#7000 WriteAndFlushEnrichedTraces net6.0 712μs 3.24μs 12.1μs 0 0 0 42.36 KB
#7000 WriteAndFlushEnrichedTraces netcoreapp3.1 650μs 3.16μs 13.8μs 0 0 0 42.15 KB
#7000 WriteAndFlushEnrichedTraces net472 907μs 2.34μs 8.74μs 4.46 0 0 55.59 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net6.0 1.93μs 7.28ns 27.2ns 0 0 0 1.03 KB
master ExecuteNonQuery netcoreapp3.1 2.42μs 10.5ns 40.6ns 0 0 0 1.02 KB
master ExecuteNonQuery net472 2.99μs 5.01ns 19.4ns 0.148 0.0148 0 995 B
#7000 ExecuteNonQuery net6.0 2.02μs 5.54ns 21.5ns 0 0 0 1.03 KB
#7000 ExecuteNonQuery netcoreapp3.1 2.46μs 2.48ns 9.62ns 0 0 0 1.02 KB
#7000 ExecuteNonQuery net472 2.7μs 2.62ns 10.2ns 0.149 0.0136 0 995 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net6.0 1.81μs 8.8ns 37.3ns 0 0 0 1.04 KB
master CallElasticsearch netcoreapp3.1 2.28μs 10.8ns 41.8ns 0 0 0 1.04 KB
master CallElasticsearch net472 3.45μs 0.785ns 2.83ns 0.156 0 0 1.05 KB
master CallElasticsearchAsync net6.0 1.83μs 7.5ns 29ns 0 0 0 1.02 KB
master CallElasticsearchAsync netcoreapp3.1 2.4μs 3.61ns 14ns 0 0 0 1.09 KB
master CallElasticsearchAsync net472 3.6μs 1.31ns 5.08ns 0.163 0 0 1.11 KB
#7000 CallElasticsearch net6.0 1.79μs 6.53ns 25.3ns 0 0 0 1.04 KB
#7000 CallElasticsearch netcoreapp3.1 2.29μs 11.5ns 48.9ns 0 0 0 1.04 KB
#7000 CallElasticsearch net472 3.48μs 2.34ns 9.08ns 0.157 0 0 1.05 KB
#7000 CallElasticsearchAsync net6.0 1.86μs 4.09ns 15.9ns 0 0 0 1.02 KB
#7000 CallElasticsearchAsync netcoreapp3.1 2.4μs 7.9ns 30.6ns 0 0 0 1.09 KB
#7000 CallElasticsearchAsync net472 3.7μs 1.08ns 4.19ns 0.167 0 0 1.11 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net6.0 1.74μs 8.76ns 39.2ns 0 0 0 960 B
master ExecuteAsync netcoreapp3.1 2.34μs 6.97ns 26.1ns 0 0 0 960 B
master ExecuteAsync net472 2.7μs 3.77ns 14.6ns 0.135 0 0 923 B
#7000 ExecuteAsync net6.0 1.81μs 9.26ns 43.5ns 0 0 0 960 B
#7000 ExecuteAsync netcoreapp3.1 2.4μs 10.4ns 40.2ns 0 0 0 960 B
#7000 ExecuteAsync net472 2.6μs 4.08ns 15.8ns 0.142 0 0 923 B
Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net6.0 6.68μs 3.97ns 15.4ns 0 0 0 2.37 KB
master SendAsync netcoreapp3.1 8.79μs 18.5ns 66.9ns 0 0 0 2.9 KB
master SendAsync net472 12.5μs 4.82ns 18ns 0.501 0 0 3.19 KB
#7000 SendAsync net6.0 7μs 6.81ns 25.5ns 0 0 0 2.37 KB
#7000 SendAsync netcoreapp3.1 8.51μs 20.8ns 80.7ns 0 0 0 2.9 KB
#7000 SendAsync net472 12.5μs 7.24ns 28ns 0.503 0 0 3.19 KB
Benchmarks.Trace.Iast.StringAspectsBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #7000

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑net472 286.72 KB 409.62 KB 122.9 KB 42.87%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑net6.0 323.37 KB 340.88 KB 17.51 KB 5.42%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑net6.0 43.1 KB 44.29 KB 1.18 KB 2.75%

Fewer allocations 🎉 in #7000

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑netcoreapp3.1 44.15 KB 42.94 KB -1.21 KB -2.74%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StringConcatBenchmark net6.0 42.5μs 203ns 1.06μs 0 0 0 43.1 KB
master StringConcatBenchmark netcoreapp3.1 49.3μs 288ns 2.27μs 0 0 0 44.15 KB
master StringConcatBenchmark net472 57.3μs 148ns 552ns 0 0 0 57.34 KB
master StringConcatAspectBenchmark net6.0 1.25ms 2.33μs 8.09μs 0 0 0 323.37 KB
master StringConcatAspectBenchmark netcoreapp3.1 470μs 5.4μs 52.4μs 0 0 0 272.8 KB
master StringConcatAspectBenchmark net472 407μs 2.03μs 8.37μs 0 0 0 286.72 KB
#7000 StringConcatBenchmark net6.0 44.4μs 232ns 1.41μs 0 0 0 44.29 KB
#7000 StringConcatBenchmark netcoreapp3.1 49.8μs 278ns 2.15μs 0 0 0 42.94 KB
#7000 StringConcatBenchmark net472 57.3μs 170ns 638ns 0 0 0 57.34 KB
#7000 StringConcatAspectBenchmark net6.0 1.39ms 2.28μs 8.21μs 0 0 0 340.88 KB
#7000 StringConcatAspectBenchmark netcoreapp3.1 533μs 1.02μs 3.66μs 0 0 0 273.86 KB
#7000 StringConcatAspectBenchmark net472 405μs 2.32μs 17.2μs 0 0 0 409.62 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.61μs 13.9ns 70.8ns 0 0 0 1.76 KB
master EnrichedLog netcoreapp3.1 3.39μs 17ns 79.9ns 0 0 0 1.76 KB
master EnrichedLog net472 3.98μs 2.8ns 10.8ns 0.262 0 0 1.69 KB
#7000 EnrichedLog net6.0 2.57μs 1.34ns 4.85ns 0 0 0 1.76 KB
#7000 EnrichedLog netcoreapp3.1 3.51μs 17.3ns 75.5ns 0 0 0 1.76 KB
#7000 EnrichedLog net472 3.95μs 2.78ns 10.4ns 0.255 0 0 1.69 KB
Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 123μs 120ns 448ns 0 0 0 4.37 KB
master EnrichedLog netcoreapp3.1 127μs 229ns 885ns 0 0 0 4.37 KB
master EnrichedLog net472 166μs 74.5ns 258ns 0 0 0 4.57 KB
#7000 EnrichedLog net6.0 122μs 220ns 851ns 0 0 0 4.37 KB
#7000 EnrichedLog netcoreapp3.1 128μs 85.4ns 308ns 0 0 0 4.37 KB
#7000 EnrichedLog net472 166μs 175ns 679ns 0 0 0 4.57 KB
Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 4.91μs 9.95ns 38.5ns 0 0 0 2.32 KB
master EnrichedLog netcoreapp3.1 6.74μs 22.4ns 86.9ns 0 0 0 2.32 KB
master EnrichedLog net472 7.42μs 5.34ns 20ns 0.335 0 0 2.14 KB
#7000 EnrichedLog net6.0 4.95μs 15.8ns 61.3ns 0 0 0 2.32 KB
#7000 EnrichedLog netcoreapp3.1 6.7μs 26.6ns 103ns 0 0 0 2.32 KB
#7000 EnrichedLog net472 7.39μs 6.25ns 24.2ns 0.331 0 0 2.14 KB
Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net6.0 2.01μs 7.73ns 29.9ns 0 0 0 1.21 KB
master SendReceive netcoreapp3.1 2.52μs 12.1ns 50ns 0 0 0 1.21 KB
master SendReceive net472 3.03μs 2.7ns 10.1ns 0.182 0 0 1.21 KB
#7000 SendReceive net6.0 1.99μs 2.79ns 10.8ns 0 0 0 1.21 KB
#7000 SendReceive netcoreapp3.1 2.58μs 5.67ns 21.9ns 0 0 0 1.21 KB
#7000 SendReceive net472 3.11μs 5.81ns 22.5ns 0.185 0 0 1.21 KB
Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 4.2μs 16.8ns 62.7ns 0 0 0 1.64 KB
master EnrichedLog netcoreapp3.1 5.65μs 15ns 58.2ns 0 0 0 1.69 KB
master EnrichedLog net472 6.63μs 6.7ns 25.9ns 0.33 0 0 2.08 KB
#7000 EnrichedLog net6.0 4.14μs 2.17ns 8.14ns 0 0 0 1.64 KB
#7000 EnrichedLog netcoreapp3.1 5.59μs 8.14ns 31.5ns 0 0 0 1.69 KB
#7000 EnrichedLog net472 6.52μs 3.52ns 13.2ns 0.326 0 0 2.08 KB
Benchmarks.Trace.SpanBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net6.0 740ns 3.57ns 13.8ns 0 0 0 584 B
master StartFinishSpan netcoreapp3.1 946ns 4.71ns 18.3ns 0 0 0 584 B
master StartFinishSpan net472 925ns 0.0692ns 0.25ns 0.0885 0 0 586 B
master StartFinishScope net6.0 899ns 1.41ns 5.46ns 0 0 0 704 B
master StartFinishScope netcoreapp3.1 1.18μs 2.84ns 11ns 0 0 0 704 B
master StartFinishScope net472 1.15μs 0.973ns 3.77ns 0.104 0 0 666 B
#7000 StartFinishSpan net6.0 733ns 3.43ns 15.4ns 0 0 0 584 B
#7000 StartFinishSpan netcoreapp3.1 945ns 4.77ns 20.8ns 0 0 0 584 B
#7000 StartFinishSpan net472 923ns 0.148ns 0.533ns 0.093 0 0 586 B
#7000 StartFinishScope net6.0 875ns 4.5ns 20.1ns 0 0 0 704 B
#7000 StartFinishScope netcoreapp3.1 1.19μs 6.28ns 31.4ns 0 0 0 704 B
#7000 StartFinishScope net472 1.14μs 0.254ns 0.951ns 0.102 0 0 666 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net6.0 1.01μs 4.85ns 20ns 0 0 0 704 B
master RunOnMethodBegin netcoreapp3.1 1.39μs 1.69ns 6.55ns 0 0 0 704 B
master RunOnMethodBegin net472 1.34μs 1.5ns 5.8ns 0.1 0 0 666 B
#7000 RunOnMethodBegin net6.0 1.01μs 4.86ns 20.6ns 0 0 0 704 B
#7000 RunOnMethodBegin netcoreapp3.1 1.43μs 6.14ns 23.8ns 0 0 0 704 B
#7000 RunOnMethodBegin net472 1.36μs 1.21ns 4.7ns 0.102 0 0 666 B

@andrewlock
Copy link
Member

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

  • Welch test with statistical test for significance of 5%
  • Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.8) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (7000) - mean (68ms)  : 66, 71
     .   : milestone, 68,
    master - mean (68ms)  : 66, 70
     .   : milestone, 68,

    section CallTarget+Inlining+NGEN
    This PR (7000) - mean (1,007ms)  : 985, 1028
     .   : milestone, 1007,
    master - mean (1,007ms)  : 987, 1027
     .   : milestone, 1007,
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (7000) - mean (102ms)  : 100, 104
     .   : milestone, 102,
    master - mean (102ms)  : 100, 104
     .   : milestone, 102,

    section CallTarget+Inlining+NGEN
    This PR (7000) - mean (694ms)  : 676, 711
     .   : milestone, 694,
    master - mean (694ms)  : 676, 712
     .   : milestone, 694,
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (7000) - mean (89ms)  : 87, 91
     .   : milestone, 89,
    master - mean (89ms)  : 87, 91
     .   : milestone, 89,

    section CallTarget+Inlining+NGEN
    This PR (7000) - mean (654ms)  : 632, 676
     .   : milestone, 654,
    master - mean (656ms)  : 633, 679
     .   : milestone, 656,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.8) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (7000) - mean (190ms)  : 186, 194
     .   : milestone, 190,
    master - mean (190ms)  : 187, 193
     .   : milestone, 190,

    section CallTarget+Inlining+NGEN
    This PR (7000) - mean (1,110ms)  : 1079, 1140
     .   : milestone, 1110,
    master - mean (1,119ms)  : 1086, 1153
     .   : milestone, 1119,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (7000) - mean (268ms)  : 264, 271
     .   : milestone, 268,
    master - mean (268ms)  : 264, 272
     .   : milestone, 268,

    section CallTarget+Inlining+NGEN
    This PR (7000) - mean (877ms)  : 846, 907
     .   : milestone, 877,
    master - mean (877ms)  : 846, 907
     .   : milestone, 877,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (7000) - mean (260ms)  : 256, 264
     .   : milestone, 260,
    master - mean (261ms)  : 258, 264
     .   : milestone, 261,

    section CallTarget+Inlining+NGEN
    This PR (7000) - mean (865ms)  : 834, 895
     .   : milestone, 865,
    master - mean (867ms)  : 845, 889
     .   : milestone, 867,
Loading

OmerRaviv
OmerRaviv approved these changes May 23, 2025
View reviewed changes
Copy link
Collaborator

@OmerRaviv OmerRaviv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dudikeleti dudikeleti merged commit 2172c93 into master May 23, 2025
132 checks passed
@dudikeleti dudikeleti deleted the dudik/skip-debugger-flaky-test branch May 23, 2025 10:21
@github-actions github-actions bot added this to the vNext-v3 milestone May 23, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@OmerRaviv OmerRaviv OmerRaviv approved these changes

@andrewlock andrewlock andrewlock approved these changes

Assignees
No one assigned
Labels
area:tests unit tests, integration tests
Projects
None yet
Milestone
3.17.0
Development

Successfully merging this pull request may close these issues.
3 participants
@dudikeleti @andrewlock @OmerRaviv