[SR-216] Increase test coverage on Linux

[gribozavr]

Previous ID	SR-216
Radar	rdar://problem/19648117
Original Reporter	@gribozavr
Type	Task

Additional Detail from JIRA

Votes	0
Component/s	Compiler, Standard Library
Labels	Task, Linux, StarterBug
Assignee	None
Priority	Medium

md5: 66b0885b16ee37449ebb1fc7351185e9

is blocked by:

• SR-417 Case of printed UUIDs differs in a platform dependent way

relates to:

• SR-451 Add protocol mirror test back to Runtime.swift test

Issue Description:

Many important tests are XFAIL'ed on Linux. Usually this is because one or few parts of the test use the Objective-C runtime or rely on Foundation.

These parts can usually be separated into separate files, or guarded with `#if _runtime(_ObjC)`. Sometimes these failures are real issues that we are unaware of and need to start tracking in the bug tracker.

It is easy to find disabled tests with:

$ git grep 'XFAIL: linux'

I consider the following tests to be of high value to be looked at first.

Standard library:

test/1_stdlib/Print.swift
test/1_stdlib/ReflectionHashing.swift
test/1_stdlib/Runtime.swift
validation-test/stdlib/Dictionary.swift
validation-test/stdlib/Set.swift
validation-test/stdlib/String.swift
validation-test/stdlib/Unicode.swift
validation-test/stdlib/UnicodeTrie.swift.gyb

SIL and the optimizer:

test/Serialization/basic_sil.swift

[Update] These ones are done:

test/1_stdlib/ArrayTraps.swift.gyb
test/1_stdlib/SetTraps.swift
test/1_stdlib/StringTraps.swift
validation-test/stdlib/UnicodeUTFEncoders.swift

test/SIL/Parser/basic.sil
test/SILOptimizer/cse.sil
test/SILOptimizer/definite_init_diagnostics.swift
test/SILOptimizer/sil_combine.sil

[swift-ci]

Comment by Emanuel Zephir (JIRA)

List of Pull Requests:
Pull request 819 SIL/Parser/basic.sil
Pull request 832 SILOptimizer/cse.sil
Pull request 835 SILOptimizer/sil_combine.sil
Pull request 839 SILOptimizer/definite_init_diagnostics.swift
Pull request 842 Serialization/def_basic.swift
Pull request 844 1_stdlib/StringTraps.swift
Pull request 852 1_stdlib/SetTraps.swift
Pull request 855 1_stdlib/ArrayTraps.swift.gyb
Pull request 862 1_stdlib/Runtime.swift (merge ready, but needs a new test: SR-451)
Pull request 905 1_stdlib/ReflectionHashing.swift (reverted in Pull request 933 because of unstable hashing. PR has more info)

Pull request 968 Collection test input data refactoring
Pull request 978 stdlib/UnicodeUTFEncoders.swift

SR-524 & SR-525 - type checking bugs affecting the dictionary and set refactorings

Not XFAILED on Linux:

• 1_stdlib/Print.swift

Need to be redone to fix merge conflicts:
Pull request 969 stdlib/Dictionary.swift
Pull request 970 stdlib/Set.swift
Pull request 972 stdlib/Unicode.swift
Pull request 977 stdlib/UnicodeTrie.swift.gyb

[swift-ci]

Comment by Félix Fischer (JIRA)

Updated the original post to show what has already been covered 🙂

[AnthonyLatsis]

The issues with these tests are either stale or resolved. Here is a resolution table for those that are not marked as been dealt with in the original post.

test/Serialization/basic_sil.swift	XFAIL removed in #842
test/1_stdlib/Print.swift Present-day test/stdlib/Print.swift	Appears to have never been XFAILed on Linux
test/1_stdlib/ReflectionHashing.swift Present-day test/stdlib/ReflectionHashing.swift	XFAIL removed in #4867
test/1_stdlib/Runtime.swift Present-day test/stdlib/Runtime.swift.gyb	XFAIL removed in #862
validation-test/stdlib/Dictionary.swift	XFAIL removed in #2544
validation-test/stdlib/Set.swift	XFAIL removed in #2561
validation-test/stdlib/String.swift	XFAIL removed in #1111
validation-test/stdlib/Unicode.swift Present-day validation-test/stdlib/Unicode.swift.gyb	XFAIL removed in #2544
validation-test/stdlib/UnicodeTrie.swift.gyb	Deleted in #11409