Enable Whisper

[jackzhxng]

🚀 The feature, motivation and pitch

• Fix current Whisper export bug on Optimum ET
• Integrate with runner / apps

Alternatives

No response

Additional context

No response

RFC (Optional)

No response

[guangy10]

Will also need to combine multiple PTEs into one

[jackzhxng]

We can't do that easily because of the cross attention

[jackzhxng]

Current whisper seems to be updating KV cache everytime - https://github.com/huggingface/optimum-executorch/blob/main/tests/models/test_modeling_whisper.py#L60-L86. Will need either a torch.cond here or need to export the model with KV cache as IO (preferred)

@chmjkb you mentioned offline that you were using your own non-Optimum export for Whisper, curious to hear if you did something different

[cccclai]

Seems like duplicate with https://github.com/pytorch/executorch/issues/12352...unless you want to target optimum specifically

[chmjkb]

Hi, we haven't done much with this export outside of what is done in Optimum. We're using a separate export script just because we implemented support for Whisper in React Native ExecuTorch before it was implemented in Optimum.