Skip to content

[AutoDiff] Add Differentiable.zeroTangentVectorInitializer. #28310

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

dan-zheng
Copy link
Contributor

The Differentiable.zeroTangentVectorInitializer requirement will enable
correct, efficient reverse-mode differentiation of struct property accesses.

A zeroTangentVectorInitializer closure is more efficient than a
zeroTangentVector computed property, which would always capture self.

Add Differentiable.zeroTangentVector default instance property, which
returns self.zeroTangentVectorInitializer().

Todos:

  • Implement derived conformances for zeroTangentVectorInitializer.
  • Implement differentiation transform support for
    zeroTangentVectorInitializer and struct projection instructions.

The `Differentiable.zeroTangentVectorInitializer` requirement will enable
correct, efficient reverse-mode differentiation of struct property accesses.

A `zeroTangentVectorInitializer` closure is more efficient than a
`zeroTangentVector` computed property, which would always capture `self`.

Add `Differentiable.zeroTangentVector` default instance property, which
returns `self.zeroTangentVectorInitializer()`.

Todos:
- Implement derived conformances for `zeroTangentVectorInitializer`.
- Implement differentiation transform support for
  `zeroTangentVectorInitializer` and struct projection instructions.
@dan-zheng dan-zheng added the tensorflow This is for "tensorflow" branch PRs. label Nov 17, 2019
@dan-zheng dan-zheng requested review from rxwei and marcrasi November 17, 2019 00:34
""")
var zeroTangentVector: TangentVector { get }
var zeroTangentVectorInitializer: () -> TangentVector { get }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Eventually this property declaration (in the protocol) should have a @noDerivative on it, and potentially require all conforming types to specify a @noDerivative ... when we support differentiable curry.


/// A tangent vector initialized using `zeroTangentVectorInitializer`.
/// `move(along: zeroTangentVector)` should not modify `self`.
var zeroTangentVector: TangentVector { zeroTangentVectorInitializer() }
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately, there's a availability warning when compiling the standard library:

Differentiable.swift:97:42: warning: 'zeroTangentVectorInitializer' is deprecated: `zeroTangentVectorInitializer` derivation has not been implemented; do not use this property
  var zeroTangentVector: TangentVector { zeroTangentVectorInitializer() }
                                         ^

I think this is acceptable for now.

@dan-zheng
Copy link
Contributor Author

@swift-ci Please test tensorflow

@dan-zheng dan-zheng merged this pull request into swiftlang:tensorflow Nov 17, 2019
@dan-zheng dan-zheng deleted the zero-tangent-initialization branch November 17, 2019 01:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
tensorflow This is for "tensorflow" branch PRs.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants