[metadata prespecialization] Cross-module: structs and enums. #32354

nate-chandler · 2020-06-12T18:36:56Z

Initial support for cross-module metadata prespecialization.

Before, given a module

M:
struct K {}
struct G<T> {}
prespecialize( G<K>.self )

(1) canonical prespecialized metadata for G<K> was defined within M

`canonical specialized generic type metadata for G<K>` = linkonce_odr hidden constant ...

(2) it was referred to directly where it was used

prespecialize(instance, &`canonical specialized generic type metadata for G<K>`)

and (3) it was returned from the metadata accessor for G

MetadataResponse `metadata accessor for G`(MetadataRequest request, Metadata *type) {
    if (type == &`type metadata for K`) {
        return `canonical specialized generic type metadata for G<K>`
    }
    return swift_getGenericMetadata(request, {type}, `nominal type descriptor for G`)
}

In particular, note that the prespecialization for G<K> had to be done within the module which defined G.

Here, support is added for creating a prespecialization in a module different from that which defines the type. Given modules

M:
struct G<T> {}
N:
import M
struct K {}
prespecialize( G<K>.self )

(1) a non-canonical prespecialized metadata for G<K> is defined within N

`noncanonical specialized generic type metadata for G<K>` = linkonce_odr hidden constant ...

and (2) it is not referred to directly from where it is used but rather passed through the new runtime function swift_getCanonicalSpecializedMetadata

prespecialize(instance, swift_getCanonicalSpecializedMetadata(&`noncanonical specialized generic type metadata for G<K>`, &`cache variable for noncanonical specialized generic type metadata for M.G<N.K>`))

The runtime call is necessary because it can't be statically guaranteed that metadata for the type described by the noncanonical prespecialization has not been created dynamically (or provided by a different noncanonical prespecialization). That might occur, for example, in the following way:

M:
func instantiate<T>(for t: T.Type) -> G<T> { ... }

N:
instantiate(K.self)
prespecialize( G<K>.self )

In this example, instantiate would be called before noncanonical specialized generic type metadata for G<K> is passed to swift_getCanonicalSpecializedMetadata. Consequently, a metadata record for G<K> would be created dynamically and added to the runtime generic metadata cache. The invariant that there only be a single, canonical metadata record for the type G<K> must be preserved. As a result, the dynamically created metadata for G<K> must be passed to prespecialize. The new runtime call ensures that: the dynamically created metadata for G<K> will be returned from

swift_getCanonicalSpecializedMetadata(&`noncanonical specialized generic type metadata for G<K>`, &`cache variable for noncanonical specialized generic type metadata for M.G<N.K>`)

Dependent on #32506

nate-chandler · 2020-06-12T18:37:05Z

@swift-ci please test

swift-ci · 2020-06-12T20:27:13Z

Build failed
Swift Test OS X Platform
Git Sha - 738f6827280d907ef2f9273e2406b83e9bdc3643

nate-chandler · 2020-06-12T23:20:00Z

@swift-ci please test

nate-chandler · 2020-06-12T23:20:36Z

@swift-ci please benchmark

swift-ci · 2020-06-12T23:22:48Z

Build failed
Swift Test Linux Platform
Git Sha - 738f6827280d907ef2f9273e2406b83e9bdc3643

swift-ci · 2020-06-12T23:23:07Z

Build failed
Swift Test OS X Platform
Git Sha - 738f6827280d907ef2f9273e2406b83e9bdc3643

swift-ci · 2020-06-13T00:23:32Z

Performance: -O

Regression	OLD	NEW	DELTA	RATIO
FlattenListLoop	1634	2464	+50.8%	0.66x (?)
FlattenListFlatMap	5354	6638	+24.0%	0.81x (?)
ArrayInitFromSlice	447	519	+16.1%	0.86x (?)
DictionaryBridgeToObjC_Access	762	866	+13.6%	0.88x (?)

Code size: -O

Performance: -Osize

Regression	OLD	NEW	DELTA	RATIO
StringBuilderWithLongSubstring	1710	2000	+17.0%	0.86x (?)
UTF8Decode_InitFromCustom_contiguous_ascii_as_ascii	382	432	+13.1%	0.88x (?)
UTF8Decode_InitFromCustom_noncontiguous_ascii_as_ascii	1128	1265	+12.1%	0.89x (?)
StringBuilderLong	1430	1550	+8.4%	0.92x (?)
DictionaryOfAnyHashableStrings_lookup	4032	4368	+8.3%	0.92x (?)

Improvement	OLD	NEW	DELTA	RATIO
DataAppendArray	4400	4100	-6.8%	1.07x (?)

Code size: -Osize

Performance: -Onone

Regression	OLD	NEW	DELTA	RATIO
UTF8Decode_InitFromCustom_contiguous_ascii	347	397	+14.4%	0.87x (?)
DictionaryCompactMapValuesOfCastValue	53730	60696	+13.0%	0.89x (?)
ArrayInitFromSlice	504	562	+11.5%	0.90x (?)
UTF8Decode_InitDecoding_ascii	329	365	+10.9%	0.90x (?)
ArrayPlusEqualThreeElements	6260	6910	+10.4%	0.91x (?)
UTF8Decode_InitFromCustom_contiguous_ascii_as_ascii	426	465	+9.2%	0.92x (?)

Code size: -swiftlibs

How to read the data

The tables contain differences in performance which are larger than 8% and differences in code size which are larger than 1%.

If you see any unexpected regressions, you should consider fixing the
regressions before you merge the PR.

Noise: Sometimes the performance results (not code size!) contain false
alarms. Unexpected regressions which are marked with '(?)' are probably noise.
If you see regressions which you cannot explain you can try to run the
benchmarks again. If regressions still show up, please consult with the
performance team (@eeckstein).

Hardware Overview

  Model Name: Mac Pro
  Model Identifier: MacPro6,1
  Processor Name: 12-Core Intel Xeon E5
  Processor Speed: 2.7 GHz
  Number of Processors: 1
  Total Number of Cores: 12
  L2 Cache (per Core): 256 KB
  L3 Cache: 30 MB
  Memory: 64 GB

nate-chandler · 2020-06-15T16:42:00Z

@swift-ci please clean test os x platform

nate-chandler · 2020-06-15T22:43:45Z

@swift-ci please test

swift-ci · 2020-06-15T22:46:08Z

Build failed
Swift Test Linux Platform
Git Sha - 8fa0926303b7ec0d94516bb7626e459771ae84b9

swift-ci · 2020-06-15T22:46:54Z

Build failed
Swift Test OS X Platform
Git Sha - 8fa0926303b7ec0d94516bb7626e459771ae84b9

nate-chandler · 2020-06-15T23:01:51Z

@swift-ci please clean test os x platform

nate-chandler · 2020-06-16T18:05:00Z

@swift-ci please test

swift-ci · 2020-06-16T19:26:59Z

Build failed
Swift Test OS X Platform
Git Sha - 9606f8211eaadd6d92012d8c6bff77acebb0ff3d

nate-chandler · 2020-06-16T21:57:35Z

@swift-ci please test os x platform

swift-ci · 2020-06-16T23:27:09Z

Build failed
Swift Test OS X Platform
Git Sha - 9606f8211eaadd6d92012d8c6bff77acebb0ff3d

nate-chandler · 2020-06-16T23:29:28Z

@swift-ci please clean test os x platform

nate-chandler · 2020-06-17T20:09:29Z

@swift-ci please test

swift-ci · 2020-06-17T20:12:04Z

Build failed
Swift Test Linux Platform
Git Sha - 9606f8211eaadd6d92012d8c6bff77acebb0ff3d

swift-ci · 2020-06-17T20:13:12Z

Build failed
Swift Test OS X Platform
Git Sha - 9606f8211eaadd6d92012d8c6bff77acebb0ff3d

nate-chandler · 2020-06-17T23:47:42Z

@swift-ci please test

swift-ci · 2020-06-17T23:48:26Z

Build failed
Swift Test Linux Platform
Git Sha - 8a5d138bd762285fe21f6149cf88229203ccd537

swift-ci · 2020-06-17T23:50:46Z

Build failed
Swift Test OS X Platform
Git Sha - 8a5d138bd762285fe21f6149cf88229203ccd537

nate-chandler · 2020-06-18T19:16:12Z

@swift-ci please test

swift-ci · 2020-06-18T19:19:07Z

Build failed
Swift Test Linux Platform
Git Sha - 385e34536a356db6534145b40b9941034726def0

swift-ci · 2020-06-20T00:02:01Z

Build failed
Swift Test OS X Platform
Git Sha - 6d8bbd9701be1e94507d282c8d14cda365b8709a

nate-chandler · 2020-06-20T00:24:10Z

@swift-ci please clean test os x platform

swift-ci · 2020-06-20T02:25:46Z

Build failed
Swift Test OS X Platform
Git Sha - 22e9c2d8e71ea5b52d79a5789fbeeb02f37a798e

nate-chandler · 2020-06-20T02:36:05Z

@swift-ci please clean test os x platform

swift-ci · 2020-06-20T04:40:11Z

Build failed
Swift Test OS X Platform
Git Sha - 22e9c2d8e71ea5b52d79a5789fbeeb02f37a798e

nate-chandler · 2020-06-22T03:27:00Z

@swift-ci please clean test os x platform

nate-chandler · 2020-06-22T19:57:58Z

@swift-ci please test

swift-ci · 2020-06-22T19:58:18Z

Build failed
Swift Test Linux Platform
Git Sha - 22e9c2d8e71ea5b52d79a5789fbeeb02f37a798e

swift-ci · 2020-06-22T20:01:26Z

Build failed
Swift Test OS X Platform
Git Sha - 22e9c2d8e71ea5b52d79a5789fbeeb02f37a798e

nate-chandler · 2020-06-23T03:41:31Z

@swift-ci please test

swift-ci · 2020-06-23T03:44:10Z

Build failed
Swift Test Linux Platform
Git Sha - 52f39c9fac2ffb4d930ec8db4ce1274df6c469f1

swift-ci · 2020-06-23T03:45:48Z

Build failed
Swift Test OS X Platform
Git Sha - 52f39c9fac2ffb4d930ec8db4ce1274df6c469f1

nate-chandler · 2020-07-16T19:33:29Z

@swift-ci please test

swift-ci · 2020-07-16T19:36:21Z

Build failed
Swift Test Linux Platform
Git Sha - 7e273e7062a8ffa1b5d20e2dc367af6c68a1330c

swift-ci · 2020-07-16T20:45:34Z

Build failed
Swift Test OS X Platform
Git Sha - 48834a50c93a680a901b1c628953fbc734e07e2a

When a generic type from a different module is not resilient within the current module and at least one of its arguments is from the current module, emit a non-canonical prespecialized record, and access that metadata via a call to swift_getCanonicalSpecializedMetadata, passing in the non-canonical record. rdar://problem/56996727 rdar://problem/56997022

nate-chandler · 2020-07-16T21:14:32Z

@swift-ci please test

swift-ci · 2020-07-16T21:15:08Z

Build failed
Swift Test OS X Platform
Git Sha - 48834a50c93a680a901b1c628953fbc734e07e2a

swift-ci · 2020-07-16T21:17:31Z

Build failed
Swift Test Linux Platform
Git Sha - 48834a50c93a680a901b1c628953fbc734e07e2a

nate-chandler · 2020-07-16T21:26:08Z

@swift-ci please clean test macos

nate-chandler · 2020-07-16T22:27:24Z

@swift-ci please test windows

nate-chandler · 2020-07-20T18:42:06Z

@swift-ci please test source compatibility

nate-chandler force-pushed the generic-metadata-prespecialization-components/crossmodule-structs-enums branch from 738f682 to 8fa0926 Compare June 12, 2020 23:19

nate-chandler force-pushed the generic-metadata-prespecialization-components/crossmodule-structs-enums branch from 8fa0926 to 9606f82 Compare June 15, 2020 22:42

nate-chandler force-pushed the generic-metadata-prespecialization-components/crossmodule-structs-enums branch from 9606f82 to 8a5d138 Compare June 17, 2020 20:08

nate-chandler force-pushed the generic-metadata-prespecialization-components/crossmodule-structs-enums branch from 8a5d138 to 385e345 Compare June 17, 2020 23:47

nate-chandler force-pushed the generic-metadata-prespecialization-components/crossmodule-structs-enums branch from 385e345 to 6d8bbd9 Compare June 18, 2020 19:13

nate-chandler force-pushed the generic-metadata-prespecialization-components/crossmodule-structs-enums branch from 22e9c2d to 52f39c9 Compare June 22, 2020 19:57

nate-chandler force-pushed the generic-metadata-prespecialization-components/crossmodule-structs-enums branch 2 times, most recently from f8ed98a to 7e273e7 Compare June 23, 2020 03:37

nate-chandler marked this pull request as ready for review June 23, 2020 18:10

nate-chandler force-pushed the generic-metadata-prespecialization-components/crossmodule-structs-enums branch from 7e273e7 to 48834a5 Compare July 16, 2020 19:32

nate-chandler force-pushed the generic-metadata-prespecialization-components/crossmodule-structs-enums branch from 48834a5 to d4b82ba Compare July 16, 2020 21:14

nate-chandler merged commit 98253ae into swiftlang:master Jul 21, 2020

nate-chandler deleted the generic-metadata-prespecialization-components/crossmodule-structs-enums branch July 21, 2020 15:04

[metadata prespecialization] Cross-module: structs and enums. #32354

[metadata prespecialization] Cross-module: structs and enums. #32354

Uh oh!

Conversation

nate-chandler commented Jun 12, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nate-chandler commented Jun 12, 2020

Uh oh!

swift-ci commented Jun 12, 2020

Uh oh!

nate-chandler commented Jun 12, 2020

Uh oh!

nate-chandler commented Jun 12, 2020

Uh oh!

swift-ci commented Jun 12, 2020

Uh oh!

swift-ci commented Jun 12, 2020

Uh oh!

swift-ci commented Jun 13, 2020

Performance: -O

Code size: -O

Performance: -Osize

Code size: -Osize

Performance: -Onone

Code size: -swiftlibs

Uh oh!

nate-chandler commented Jun 15, 2020

Uh oh!

nate-chandler commented Jun 15, 2020

Uh oh!

swift-ci commented Jun 15, 2020

Uh oh!

swift-ci commented Jun 15, 2020

Uh oh!

nate-chandler commented Jun 15, 2020

Uh oh!

nate-chandler commented Jun 16, 2020

Uh oh!

swift-ci commented Jun 16, 2020

Uh oh!

nate-chandler commented Jun 16, 2020

Uh oh!

swift-ci commented Jun 16, 2020

Uh oh!

nate-chandler commented Jun 16, 2020

Uh oh!

nate-chandler commented Jun 17, 2020

Uh oh!

swift-ci commented Jun 17, 2020

Uh oh!

swift-ci commented Jun 17, 2020

Uh oh!

nate-chandler commented Jun 17, 2020

Uh oh!

swift-ci commented Jun 17, 2020

Uh oh!

swift-ci commented Jun 17, 2020

Uh oh!

nate-chandler commented Jun 18, 2020

Uh oh!

swift-ci commented Jun 18, 2020

Uh oh!

swift-ci commented Jun 20, 2020

Uh oh!

nate-chandler commented Jun 20, 2020

Uh oh!

swift-ci commented Jun 20, 2020

Uh oh!

nate-chandler commented Jun 20, 2020

Uh oh!

swift-ci commented Jun 20, 2020

Uh oh!

nate-chandler commented Jun 22, 2020

Uh oh!

nate-chandler commented Jun 22, 2020

Uh oh!

swift-ci commented Jun 22, 2020

Uh oh!

swift-ci commented Jun 22, 2020

nate-chandler commented Jun 12, 2020 •

edited

Loading