Skip to content

Fix arm64 SQL server not found flaky tests #6948

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 3 commits into from
May 14, 2025
Merged

Fix arm64 SQL server not found flaky tests #6948

merged 3 commits into from
May 14, 2025

Conversation

NachoEchevarria
Copy link
Contributor

@NachoEchevarria NachoEchevarria commented May 9, 2025 •
edited
Loading

Summary of changes

In some alpine, arm64 tests, some SqlServer tests are failing because the server is not found.

This PR adds the skip (13) return code to the samples, so we can skip the test and log it.

Reason for change

Implementation details

Test coverage

Other details

@github-actions github-actions bot added the area:tests unit tests, integration tests label May 9, 2025
@pr-commenter
Copy link

pr-commenter bot commented May 9, 2025

Benchmarks

Benchmarks Report for benchmark platform 🐌

Benchmarks for #99 compared to master:

  • All benchmarks have the same speed
  • 3 benchmarks have fewer allocations
  • 3 benchmarks have more allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #99

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.ActivityBenchmark.StartStopWithChild‑net6.0 5.55 KB 5.64 KB 90 B 1.62%

Fewer allocations 🎉 in #99

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.ActivityBenchmark.StartStopWithChild‑netcoreapp3.1 5.75 KB 5.72 KB -34 B -0.59%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartStopWithChild net6.0 10.8μs 39.3ns 147ns 0 0 0 5.55 KB
master StartStopWithChild netcoreapp3.1 13.5μs 69.8ns 320ns 0 0 0 5.75 KB
master StartStopWithChild net472 22.5μs 127ns 882ns 0.912 0.228 0 6.14 KB
#99 StartStopWithChild net6.0 10.8μs 61.3ns 438ns 0 0 0 5.64 KB
#99 StartStopWithChild netcoreapp3.1 14.1μs 68.3ns 273ns 0 0 0 5.72 KB
#99 StartStopWithChild net472 21.7μs 122ns 789ns 0.992 0.331 0.11 6.16 KB
Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #99

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.AgentWriterBenchmark.WriteAndFlushEnrichedTraces‑net472 3.31 KB 3.35 KB 46 B 1.39%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 927μs 49.2ns 184ns 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 1.01ms 182ns 680ns 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces net472 1.24ms 81.7ns 316ns 0 0 0 3.31 KB
#99 WriteAndFlushEnrichedTraces net6.0 947μs 63.7ns 238ns 0 0 0 2.7 KB
#99 WriteAndFlushEnrichedTraces netcoreapp3.1 1.02ms 92.7ns 321ns 0 0 0 2.7 KB
#99 WriteAndFlushEnrichedTraces net472 1.23ms 842ns 3.26μs 0 0 0 3.35 KB
Benchmarks.Trace.Asm.AppSecBodyBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net6.0 327μs 1.54μs 6.52μs 0 0 0 196.71 KB
master AllCycleSimpleBody netcoreapp3.1 499μs 1.66μs 5.98μs 0 0 0 204.37 KB
master AllCycleSimpleBody net472 438μs 112ns 418ns 36.6 2.16 0 235.77 KB
master AllCycleMoreComplexBody net6.0 336μs 1.41μs 5.44μs 0 0 0 200.21 KB
master AllCycleMoreComplexBody netcoreapp3.1 480μs 1.79μs 6.47μs 0 0 0 207.78 KB
master AllCycleMoreComplexBody net472 449μs 149ns 576ns 37.9 2.23 0 239.27 KB
master ObjectExtractorSimpleBody net6.0 341ns 1.34ns 5ns 0 0 0 280 B
master ObjectExtractorSimpleBody netcoreapp3.1 395ns 2.16ns 11.6ns 0 0 0 272 B
master ObjectExtractorSimpleBody net472 300ns 0.0237ns 0.0853ns 0.0439 0 0 281 B
master ObjectExtractorMoreComplexBody net6.0 6.28μs 31.2ns 117ns 0 0 0 3.78 KB
master ObjectExtractorMoreComplexBody netcoreapp3.1 7.74μs 30.5ns 118ns 0 0 0 3.69 KB
master ObjectExtractorMoreComplexBody net472 6.68μs 1.92ns 7.19ns 0.6 0 0 3.8 KB
#99 AllCycleSimpleBody net6.0 328μs 174ns 673ns 0 0 0 196.71 KB
#99 AllCycleSimpleBody netcoreapp3.1 479μs 1.64μs 6.34μs 0 0 0 204.37 KB
#99 AllCycleSimpleBody net472 438μs 139ns 520ns 36.6 2.16 0 235.77 KB
#99 AllCycleMoreComplexBody net6.0 330μs 1.71μs 8.21μs 0 0 0 200.21 KB
#99 AllCycleMoreComplexBody netcoreapp3.1 510μs 440ns 1.7μs 0 0 0 207.78 KB
#99 AllCycleMoreComplexBody net472 450μs 136ns 527ns 37.9 2.23 0 239.27 KB
#99 ObjectExtractorSimpleBody net6.0 306ns 1.73ns 11.5ns 0 0 0 280 B
#99 ObjectExtractorSimpleBody netcoreapp3.1 401ns 2.04ns 9.79ns 0 0 0 272 B
#99 ObjectExtractorSimpleBody net472 303ns 0.0233ns 0.087ns 0.0441 0 0 281 B
#99 ObjectExtractorMoreComplexBody net6.0 6.31μs 3.09ns 11.6ns 0 0 0 3.78 KB
#99 ObjectExtractorMoreComplexBody netcoreapp3.1 7.76μs 29ns 112ns 0 0 0 3.69 KB
#99 ObjectExtractorMoreComplexBody net472 6.76μs 1.97ns 7.64ns 0.575 0 0 3.8 KB
Benchmarks.Trace.Asm.AppSecEncoderBenchmark - Unknown 🤷 Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EncodeArgs net6.0 N/A N/A N/A NaN NaN NaN 0 b
master EncodeArgs netcoreapp3.1 N/A N/A N/A NaN NaN NaN 0 b
master EncodeArgs net472 N/A N/A N/A NaN NaN NaN 0 b
master EncodeLegacyArgs net6.0 N/A N/A N/A NaN NaN NaN 0 b
master EncodeLegacyArgs netcoreapp3.1 N/A N/A N/A NaN NaN NaN 0 b
master EncodeLegacyArgs net472 N/A N/A N/A NaN NaN NaN 0 b
#99 EncodeArgs net6.0 N/A N/A N/A NaN NaN NaN 0 b
#99 EncodeArgs netcoreapp3.1 N/A N/A N/A NaN NaN NaN 0 b
#99 EncodeArgs net472 N/A N/A N/A NaN NaN NaN 0 b
#99 EncodeLegacyArgs net6.0 N/A N/A N/A NaN NaN NaN 0 b
#99 EncodeLegacyArgs netcoreapp3.1 N/A N/A N/A NaN NaN NaN 0 b
#99 EncodeLegacyArgs net472 N/A N/A N/A NaN NaN NaN 0 b
Benchmarks.Trace.Asm.AppSecWafBenchmark - Unknown 🤷 Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunWafRealisticBenchmark net6.0 N/A N/A N/A NaN NaN NaN 0 b
master RunWafRealisticBenchmark netcoreapp3.1 N/A N/A N/A NaN NaN NaN 0 b
master RunWafRealisticBenchmark net472 N/A N/A N/A NaN NaN NaN 0 b
master RunWafRealisticBenchmarkWithAttack net6.0 N/A N/A N/A NaN NaN NaN 0 b
master RunWafRealisticBenchmarkWithAttack netcoreapp3.1 N/A N/A N/A NaN NaN NaN 0 b
master RunWafRealisticBenchmarkWithAttack net472 N/A N/A N/A NaN NaN NaN 0 b
#99 RunWafRealisticBenchmark net6.0 N/A N/A N/A NaN NaN NaN 0 b
#99 RunWafRealisticBenchmark netcoreapp3.1 N/A N/A N/A NaN NaN NaN 0 b
#99 RunWafRealisticBenchmark net472 N/A N/A N/A NaN NaN NaN 0 b
#99 RunWafRealisticBenchmarkWithAttack net6.0 N/A N/A N/A NaN NaN NaN 0 b
#99 RunWafRealisticBenchmarkWithAttack netcoreapp3.1 N/A N/A N/A NaN NaN NaN 0 b
#99 RunWafRealisticBenchmarkWithAttack net472 N/A N/A N/A NaN NaN NaN 0 b
Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net6.0 61.7μs 39.1ns 141ns 0 0 0 14.53 KB
master SendRequest netcoreapp3.1 72.2μs 117ns 420ns 0 0 0 17.42 KB
master SendRequest net472 0.0166ns 0.00188ns 0.00729ns 0 0 0 0 b
#99 SendRequest net6.0 60μs 38.5ns 144ns 0 0 0 14.53 KB
#99 SendRequest netcoreapp3.1 71.4μs 376ns 1.88μs 0 0 0 17.42 KB
#99 SendRequest net472 0.0136ns 0.00122ns 0.00471ns 0 0 0 0 b
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 684μs 3.86μs 25.6μs 0 0 0 41.66 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 696μs 3.97μs 30.5μs 0 0 0 41.72 KB
master WriteAndFlushEnrichedTraces net472 844μs 3.8μs 14.7μs 4.46 0 0 55.75 KB
#99 WriteAndFlushEnrichedTraces net6.0 663μs 3.03μs 11.7μs 0 0 0 41.72 KB
#99 WriteAndFlushEnrichedTraces netcoreapp3.1 738μs 4.26μs 34.3μs 0 0 0 41.87 KB
#99 WriteAndFlushEnrichedTraces net472 921μs 2.08μs 7.79μs 4.46 0 0 56.03 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net6.0 1.97μs 6.61ns 25.6ns 0 0 0 1.03 KB
master ExecuteNonQuery netcoreapp3.1 2.47μs 7.74ns 30ns 0 0 0 1.02 KB
master ExecuteNonQuery net472 2.82μs 3.29ns 12.7ns 0.155 0.0141 0 995 B
#99 ExecuteNonQuery net6.0 1.97μs 6.02ns 22.5ns 0 0 0 1.03 KB
#99 ExecuteNonQuery netcoreapp3.1 2.48μs 11.9ns 47.7ns 0 0 0 1.02 KB
#99 ExecuteNonQuery net472 2.8μs 4.55ns 17.6ns 0.154 0.014 0 995 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net6.0 1.87μs 5.16ns 20ns 0 0 0 1.04 KB
master CallElasticsearch netcoreapp3.1 2.26μs 5.72ns 22.1ns 0 0 0 1.04 KB
master CallElasticsearch net472 3.55μs 4.21ns 16.3ns 0.159 0 0 1.05 KB
master CallElasticsearchAsync net6.0 1.93μs 4.48ns 17.4ns 0 0 0 1.02 KB
master CallElasticsearchAsync netcoreapp3.1 2.29μs 9.98ns 37.4ns 0 0 0 1.09 KB
master CallElasticsearchAsync net472 3.6μs 1.66ns 6.42ns 0.163 0 0 1.11 KB
#99 CallElasticsearch net6.0 1.77μs 9.49ns 48.4ns 0 0 0 1.04 KB
#99 CallElasticsearch netcoreapp3.1 2.37μs 10.6ns 41ns 0 0 0 1.04 KB
#99 CallElasticsearch net472 3.5μs 1.71ns 6.63ns 0.157 0 0 1.05 KB
#99 CallElasticsearchAsync net6.0 1.81μs 2.81ns 10.1ns 0 0 0 1.02 KB
#99 CallElasticsearchAsync netcoreapp3.1 2.46μs 10.9ns 42.1ns 0 0 0 1.09 KB
#99 CallElasticsearchAsync net472 3.65μs 4.76ns 18.5ns 0.163 0 0 1.11 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net6.0 1.78μs 1.91ns 7.38ns 0 0 0 960 B
master ExecuteAsync netcoreapp3.1 2.34μs 9.99ns 38.7ns 0 0 0 960 B
master ExecuteAsync net472 2.55μs 1.01ns 3.91ns 0.14 0 0 923 B
#99 ExecuteAsync net6.0 1.77μs 9.1ns 44.6ns 0 0 0 960 B
#99 ExecuteAsync netcoreapp3.1 2.29μs 3.85ns 14.9ns 0 0 0 960 B
#99 ExecuteAsync net472 2.59μs 0.949ns 3.67ns 0.142 0 0 923 B
Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net6.0 7.1μs 8.11ns 30.3ns 0 0 0 2.37 KB
master SendAsync netcoreapp3.1 8.4μs 24.3ns 90.9ns 0 0 0 2.9 KB
master SendAsync net472 12.6μs 11.2ns 43.5ns 0.501 0 0 3.19 KB
#99 SendAsync net6.0 7.16μs 9.76ns 36.5ns 0 0 0 2.37 KB
#99 SendAsync netcoreapp3.1 8.66μs 4.43ns 16ns 0 0 0 2.9 KB
#99 SendAsync net472 12.3μs 7.29ns 28.2ns 0.498 0 0 3.19 KB
Benchmarks.Trace.Iast.StringAspectsBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #99

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑netcoreapp3.1 250.58 KB 256.52 KB 5.94 KB 2.37%

Fewer allocations 🎉 in #99

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑net6.0 43.53 KB 42.86 KB -672 B -1.54%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑net472 65.54 KB 57.34 KB -8.19 KB -12.50%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StringConcatBenchmark net6.0 41.8μs 199ns 746ns 0 0 0 43.53 KB
master StringConcatBenchmark netcoreapp3.1 54.6μs 824ns 8.07μs 0 0 0 42.86 KB
master StringConcatBenchmark net472 57.4μs 237ns 885ns 0 0 0 65.54 KB
master StringConcatAspectBenchmark net6.0 498μs 1.64μs 5.93μs 0 0 0 276.38 KB
master StringConcatAspectBenchmark netcoreapp3.1 514μs 1.48μs 5.52μs 0 0 0 250.58 KB
master StringConcatAspectBenchmark net472 406μs 2.33μs 17.7μs 0 0 0 278.53 KB
#99 StringConcatBenchmark net6.0 42μs 224ns 1.1μs 0 0 0 42.86 KB
#99 StringConcatBenchmark netcoreapp3.1 51.3μs 441ns 4.14μs 0 0 0 42.84 KB
#99 StringConcatBenchmark net472 56.8μs 235ns 911ns 0 0 0 57.34 KB
#99 StringConcatAspectBenchmark net6.0 490μs 1.39μs 5.02μs 0 0 0 277.24 KB
#99 StringConcatAspectBenchmark netcoreapp3.1 516μs 1.64μs 5.9μs 0 0 0 256.52 KB
#99 StringConcatAspectBenchmark net472 402μs 2μs 15.9μs 0 0 0 278.53 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.55μs 5.21ns 19.5ns 0 0 0 1.76 KB
master EnrichedLog netcoreapp3.1 3.47μs 16.6ns 66.3ns 0 0 0 1.76 KB
master EnrichedLog net472 3.91μs 3.44ns 13.3ns 0.254 0 0 1.69 KB
#99 EnrichedLog net6.0 2.58μs 13.7ns 62.6ns 0 0 0 1.76 KB
#99 EnrichedLog netcoreapp3.1 3.43μs 4.1ns 15.9ns 0 0 0 1.76 KB
#99 EnrichedLog net472 3.96μs 3.62ns 14ns 0.256 0 0 1.69 KB
Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 122μs 109ns 394ns 0 0 0 4.37 KB
master EnrichedLog netcoreapp3.1 127μs 40.5ns 157ns 0 0 0 4.37 KB
master EnrichedLog net472 166μs 68.6ns 257ns 0 0 0 4.57 KB
#99 EnrichedLog net6.0 123μs 228ns 853ns 0 0 0 4.37 KB
#99 EnrichedLog netcoreapp3.1 129μs 330ns 1.24μs 0 0 0 4.37 KB
#99 EnrichedLog net472 166μs 183ns 707ns 0 0 0 4.57 KB
Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 5.13μs 4.27ns 16.5ns 0 0 0 2.32 KB
master EnrichedLog netcoreapp3.1 6.57μs 12.4ns 47.9ns 0 0 0 2.32 KB
master EnrichedLog net472 7.24μs 4.4ns 15.2ns 0.33 0 0 2.14 KB
#99 EnrichedLog net6.0 5.05μs 11.6ns 45.1ns 0 0 0 2.32 KB
#99 EnrichedLog netcoreapp3.1 6.85μs 9.07ns 35.1ns 0 0 0 2.32 KB
#99 EnrichedLog net472 7.34μs 8.4ns 30.3ns 0.33 0 0 2.14 KB
Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net6.0 2μs 10.3ns 49.5ns 0 0 0 1.21 KB
master SendReceive netcoreapp3.1 2.57μs 12.6ns 55.1ns 0 0 0 1.21 KB
master SendReceive net472 3.05μs 1.92ns 6.92ns 0.184 0 0 1.21 KB
#99 SendReceive net6.0 1.99μs 9.93ns 45.5ns 0 0 0 1.21 KB
#99 SendReceive netcoreapp3.1 2.62μs 9.24ns 35.8ns 0 0 0 1.21 KB
#99 SendReceive net472 3.11μs 1.48ns 5.75ns 0.186 0 0 1.21 KB
Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 4.11μs 11.4ns 42.6ns 0 0 0 1.64 KB
master EnrichedLog netcoreapp3.1 5.61μs 12.6ns 48.6ns 0 0 0 1.69 KB
master EnrichedLog net472 6.58μs 5.42ns 21ns 0.331 0 0 2.08 KB
#99 EnrichedLog net6.0 4.22μs 5.29ns 19.8ns 0 0 0 1.64 KB
#99 EnrichedLog netcoreapp3.1 5.5μs 21.3ns 82.5ns 0 0 0 1.69 KB
#99 EnrichedLog net472 6.63μs 7.82ns 30.3ns 0.33 0 0 2.08 KB
Benchmarks.Trace.SpanBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net6.0 744ns 4.1ns 23.2ns 0 0 0 584 B
master StartFinishSpan netcoreapp3.1 943ns 4.67ns 21.4ns 0 0 0 584 B
master StartFinishSpan net472 922ns 0.264ns 0.988ns 0.0921 0 0 586 B
master StartFinishScope net6.0 896ns 0.753ns 2.92ns 0 0 0 704 B
master StartFinishScope netcoreapp3.1 1.17μs 1.61ns 6.02ns 0 0 0 704 B
master StartFinishScope net472 1.13μs 0.222ns 0.859ns 0.101 0 0 666 B
#99 StartFinishSpan net6.0 747ns 0.214ns 0.83ns 0 0 0 584 B
#99 StartFinishSpan netcoreapp3.1 943ns 4.51ns 17.5ns 0 0 0 584 B
#99 StartFinishSpan net472 939ns 0.112ns 0.403ns 0.0899 0 0 586 B
#99 StartFinishScope net6.0 913ns 4.7ns 22.5ns 0 0 0 704 B
#99 StartFinishScope netcoreapp3.1 1.13μs 5.73ns 25.6ns 0 0 0 704 B
#99 StartFinishScope net472 1.1μs 0.162ns 0.629ns 0.105 0 0 666 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net6.0 1.08μs 1.8ns 6.98ns 0 0 0 704 B
master RunOnMethodBegin netcoreapp3.1 1.33μs 4.6ns 17.2ns 0 0 0 704 B
master RunOnMethodBegin net472 1.35μs 0.669ns 2.59ns 0.101 0 0 666 B
#99 RunOnMethodBegin net6.0 995ns 2.08ns 8.04ns 0 0 0 704 B
#99 RunOnMethodBegin netcoreapp3.1 1.38μs 6.79ns 28ns 0 0 0 704 B
#99 RunOnMethodBegin net472 1.37μs 1.08ns 4.17ns 0.101 0 0 666 B

@datadog-datadog-prod-us1
Copy link

datadog-datadog-prod-us1 bot commented May 9, 2025 •
edited
Loading

Datadog Report

All test runs e5890c6 🔗

✅ 2 Total Test Services: 0 Failed, 2 Passed

Test Services
Service Name Failed Known Flaky New Flaky Passed Skipped Total Time Test Service View
dd-trace-dotnet 0 0 0 253992 2290 17h 35m 31.99s Link
exploration_tests 0 0 0 22085 3 2m 8.78s Link

@andrewlock
Copy link
Member

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

  • Welch test with statistical test for significance of 5%
  • Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.8) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6948) - mean (68ms)  : 66, 71
     .   : milestone, 68,
    master - mean (69ms)  : 64, 75
     .   : milestone, 69,

    section CallTarget+Inlining+NGEN
    This PR (6948) - mean (1,008ms)  : 975, 1040
     .   : milestone, 1008,
    master - mean (1,004ms)  : 976, 1033
     .   : milestone, 1004,
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6948) - mean (102ms)  : 100, 105
     .   : milestone, 102,
    master - mean (103ms)  : 101, 104
     .   : milestone, 103,

    section CallTarget+Inlining+NGEN
    This PR (6948) - mean (694ms)  : 670, 718
     .   : milestone, 694,
    master - mean (693ms)  : 676, 711
     .   : milestone, 693,
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6948) - mean (90ms)  : 88, 91
     .   : milestone, 90,
    master - mean (90ms)  : 88, 92
     .   : milestone, 90,

    section CallTarget+Inlining+NGEN
    This PR (6948) - mean (654ms)  : 634, 673
     .   : milestone, 654,
    master - mean (654ms)  : 634, 674
     .   : milestone, 654,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.8) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6948) - mean (189ms)  : 183, 195
     .   : milestone, 189,
    master - mean (189ms)  : 186, 192
     .   : milestone, 189,

    section CallTarget+Inlining+NGEN
    This PR (6948) - mean (1,107ms)  : 1082, 1132
     .   : milestone, 1107,
    master - mean (1,106ms)  : 1079, 1134
     .   : milestone, 1106,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6948) - mean (269ms)  : 264, 273
     .   : milestone, 269,
    master - mean (269ms)  : 263, 275
     .   : milestone, 269,

    section CallTarget+Inlining+NGEN
    This PR (6948) - mean (882ms)  : 852, 913
     .   : milestone, 882,
    master - mean (882ms)  : 858, 905
     .   : milestone, 882,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6948) - mean (261ms)  : 256, 267
     .   : milestone, 261,
    master - mean (262ms)  : 258, 266
     .   : milestone, 262,

    section CallTarget+Inlining+NGEN
    This PR (6948) - mean (865ms)  : 839, 891
     .   : milestone, 865,
    master - mean (870ms)  : 834, 907
     .   : milestone, 870,
Loading

@NachoEchevarria NachoEchevarria changed the title Nacho/fix sql server flaky tests Fix arm64 SQL server not found flaky tests May 9, 2025
andrewlock
andrewlock approved these changes May 9, 2025
View reviewed changes
Copy link
Member

@andrewlock andrewlock left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

@NachoEchevarria NachoEchevarria marked this pull request as ready for review May 9, 2025 15:26
@NachoEchevarria NachoEchevarria requested review from a team as code owners May 9, 2025 15:26
bouwkast
bouwkast approved these changes May 13, 2025
View reviewed changes
Copy link
Collaborator

@bouwkast bouwkast left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great thanks!

@NachoEchevarria
Copy link
Contributor Author

Thanks for your reviews!

@NachoEchevarria NachoEchevarria merged commit 0e16080 into master May 14, 2025
125 of 128 checks passed
@NachoEchevarria NachoEchevarria deleted the nacho/FixSqlServerFlakyTests branch May 14, 2025 08:34
@github-actions github-actions bot added this to the vNext-v3 milestone May 14, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@bouwkast bouwkast bouwkast approved these changes

@andrewlock andrewlock andrewlock approved these changes

Assignees
No one assigned
Labels
area:tests unit tests, integration tests
Projects
None yet
Milestone
3.17.0
Development

Successfully merging this pull request may close these issues.
3 participants
@NachoEchevarria @andrewlock @bouwkast