What about subnormals?

[jfbastien]

Forking from: #141 (comment)

Should denormal be:

• Unspecified (say so in Nondeterminism.md).
• Fully specified as IEEE-754 compliance.
• Fully specified as IEEE-754 compliance for scalars, and do something for SIMD because existing hardware (especially ARMv7 NEON) doesn't support denormals.
• Specified as DAZ/FTZ (not IEEE-754 compliant).

We should probably let ourselves change this from developer feedback, but I'd like to make some decision for MVP. I suggested on esdiscuss that JavaScript go full unspecified, and just do DAZ/FTZ because it's often faster. Yes, x86 is better than it used to be but that's not universal, ignores current hardware, and doesn't look towards what new hardware will do. I like leaving the door open :-)

For @sunfishcode searchability, I'll use the word "subnormals" too :-)

[sunfishcode]

For @jfbastien searchability, I'll say the word "denormals" too :-). But IEEE754-2008 is about 7 years ago, so it's time to be up to date :-).

[titzer]

I would argue for IEEE 754 compliance from the beginning. The rationale is that hardware and software all tends toward the standard over time. I think we're uncorking a long-term bottle of annoyance and incompatibility to deviate.

As for subnormals, it seems the same cycle has repeated multiple times in floating point history. It's been tempting for hardware to cut corners to either do FTZ or something else, and has always tended back to IEEE 754 full compliance. Even GPUs are implementing full IEEE now. Of our tier 1 platforms, the only one I am aware of where subnormals are not implemented at all is Float32x4 on Arm NEON (i.e. SIMD). Float64x2 on NEON is fully IEEE compliant. Scalar arithmetic on arm is of course compliant.

The subnormal situation seems to come down to SIMD, specifically the case of Arm NEON above.

Based on conversations with hardware designers, microarchitectures have gotten so much better at superscalar floating point, even Arm cores, that it might be acceptable to spec Float32 as IEEE as well as Float32 SIMD operations as IEEE, and then simply not do Float32x4 as SIMD on arm. But this is something we should measure and motivate when SIMD is coming into WebAsm. At that point, if the performance really justifies weakening the spec, we can weaken the spec. Otherwise, it would be hard to tighten up the spec later.

[jfbastien]

My objection to that is: which developers care about denormals? Developers usually learn about them when they have stray denormals in their compute kernel and code goes orders of magnitude slower. I'm still looking for someone who wants denormal support.

[lukewagner]

If we specify FTZ as a developer-controlled mode, we're not weakening the spec (as in, we're not making semantics any looser); we're giving strictly more power to developers and this is something that they are specifically asking for (in our discussions with asm.js-using gamedevs). If we consider that wasm will always be run on emerging platforms (which often start w/ terrible denormal perf) and very old platforms, then this will be a consistent feature request, not one that will get definitely fixed. Furthermore, from my discussions with Intel, even with new, optimized denormals, they're not equivalent in speed to normal numbers and they're also not optimized for all ops. This is why setting FTZ is standard practice; it's just one less perf cliff to worry about.

[jfbastien]

Setting DAZ/FTZ should be a global property that's set once for the entire wasm application, though?

• Or can it be set/unset arbitrarily and mess with AOT compilation?
• What of threading setting/unsetting it?
• Wasm shares its process with other modules and non-wasm code. What does that imply when setting/unsetting?

[lukewagner]

I would expect it to be a global flag on a wasm module that cannot be toggled and can thus be assumed for AOT/cached compilation. That still leaves questions w/ dynamic linking but I'd default to the simple option of: if you try to dynamically link a wasm module w/ a different flag than you, loading fails.

[titzer]

I want to see data, and the burden of proof for violating IEEE 754 is very
high in my book. Even if the performance gains are huge, they'll likely be
spotty, i.e. on only a small number of platforms. I'd recommend that we
find a way to spec it in an "opt-in to nondeterminism because I want speed"
fashion. But mostly I trust the IEEE 754 specification and the numerical
expertise that went in to it more than hardware manufacturers on the
cutting edge.

I think the phase where we gather sufficient data to motivate a digression
from IEEE 754 hasn't come yet, so spec'ing nondeterminism or deviance seems
premature now.

I think mandating FTZ is a no-go, since mode-switching seems to be really
expensive on Intel.

On Thu, Jun 11, 2015 at 12:14 PM, Luke Wagner notifications@github.com
wrote:

I would expect it to be a global flag on a wasm module that cannot be
toggled and can thus be assumed for AOT/cached compilation. That still
leaves questions w/ dynamic linking but I'd default to the simple option
of: if you try to dynamically link a wasm module w/ a different flag than
you, loading fails.

—
Reply to this email directly or view it on GitHub
#148 (comment).

[lukewagner]

To be clear, it's not nondeterminism that is being discussed: it's (deterministically) flushing denormals to zero. Also, we have had reports (e.g. and these guys iirc) specifically about people hitting denormal perf problems in JS. That setting FTZ (globally, not toggling dynamically) is standard practice for whole domains (games, signal processing) demonstrates that this is something developers expect.