You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This notebook is an example of a (working) "Hacky" solution for wrapping the full 'generate' functionality inside a "forward" function.
The perpose of this is to start a discussion and be a suggention on how to make the this functionality TorchScriptable.
To do so, I:
1. T5TorchGenerative: inherited from T5Model:
- extracted the decoding code from t5.forward() function to a standalone 'decode' function that returns a specific type.
- added the GenerationUtils's 'generate' functionality as a class method (similar to HuggingFace).
2. Added TorchScriptableT5, a module that implements the full generative logic in the forward method.
3. Helper classes that build a jit (TorchScript) model from a predefined T5 Bundle
Co-authored-by: Roei Bahumi <[email protected]>
0 commit comments