Description
We can add a second parameter to decode/encode and the output callbacks (for audio, video and images), that would be respectively an output buffer or handle, and the input buffer or handle (and also mirror it for encode vs. decode), to allow authors to reuse memory and avoid lots of memory traffic. GCs are amazing, but it's always better to not GC, reuse allocations, and always use the same bit of the address space, especially in very demanding multimedia scenarios.
This is really compelling whenever the input (resp. output for decode) is regular memory, since it means the working set is going to be pretty tight (for example help with w3c/webtransport#231), but is really nice even when some buffers are backed by (say) GPU memory (because you're not doing malloc/free pairs for input/output buffers for respectively decoding and encoding). I'm not sure if GPU-memory backed surfaces can work in the same way.
Essentially, the proposal is to "pass through" the input to the output so that it can be reused. This reduces allocator use, but more importantly greatly improves memory locality.
When not using this new parameter, the implementation allocates as usual. This means that authors don't have to try to guess the buffer size, it's allocated once and then can be resized. The buffer is released naturally because it isn't referenced, immediately. In short, this is not a breaking change, but allows developers that want to control their memory allocation to be able to do so (and I believe this is something that is important for quite a few domains).
When the provided buffer is too small, the browser can allocate a new buffer, or use more complex tricks (realloc, or maybe it finds that the array buffer backing memory storage was in fact big enough, etc.).
When the provided buffer is way too big, the implementation can decide to swap it for a buffer that has a more appropriate size.