Release mode is way more slower than debug mode for user defined functions.

[nabinbhandari]

I was trying to benchmark the performance of flutter and compare it with that of Java and C. So I chose a simple function to calculate nth prime number.

The problem is that the time taken to calculate the 20000th prime number in release mode (821 ms) was almost 8 times slower than that in debug mode (106 ms).

The method that I used is given below:

  int nThPrimeNumber(int n) {
    int counter = 0;

    for (var i = 1;; i++) {
      if (isPrime(i)) counter++;

      if (counter == n) {
        return i;
      }
    }
  }

  bool isPrime(var n) {
    if (n < 2) return false;

    for (var i = 2; i * i <= n; i++) {
      if (n % i == 0) return false;
    }

    return true;
  }

Full source code and benchmark result is available in this repository.

I know this is not a big issue and there are some serious issues which needs to be resolved. But I am creating this issue because I found that in debug mode, Flutter code was running faster than C (but only in debug mode).

If this issue is fixed for release mode, some CPU intensive tasks such as image processing could be easily (and quickly too!) performed in Dart without having the need to write them in C/C++.

[mraleph]

Can you specify what kind of phone and which branch of Flutter did you use?

[nabinbhandari]

Flutter Version:

[√] Flutter (Channel beta, v0.5.1, on Microsoft Windows [Version 10.0.17134.167], locale en-US)
    • Flutter version 0.5.1 at C:\Users\Nabin\Flutter\sdk\flutter
    • Framework revision c7ea3ca377 (8 weeks ago), 2018-05-29 21:07:33 +0200
    • Engine revision 1ed25ca7b7
    • Dart version 2.0.0-dev.58.0.flutter-f981f09760

Device: Motorola Moto G5 Plus, I don't think this is device specific issue because I also found a similar issue in Samsung Galaxy A5. I haven't tested it in other devices.

One can simply verify it by running this file.

[sir-boformer]

I can confirm this on a Sony Xperia Z5 Compact:

Debug build:

I/flutter ( 1444): Flutter - 20000th prime number: 224737 
I/flutter ( 1444): time: 72 ms

Release build:

I/flutter ( 2848): Flutter - 20000th prime number: 224737 
I/flutter ( 2848): time: 383 ms

[mraleph]

I took a look: it's a combination of known issues. Here is the graph for isPrime loop in AOT mode.

 18: B9[join]:72 pred(B4, B8) {
      v7 <- phi(v3, v34 T{Type: class '_int64@0150898'}) alive [-9223372036854775808, 9223372036854775807] T{Type: class '_int64@0150898'}
}
 19:     ParallelMove S-3 <- rdx
 20:     CheckStackOverflow:78(depth 1)
 22:     PushArgument(v7)
 24:     PushArgument(v7)
 26:     v9 <- StaticCall:32( *<0> v7, v7)
 27:     ParallelMove rbx <- rax
 28:     ParallelMove S-4 <- rbx
 28:     PushArgument(v9)
 29:     ParallelMove rax <- S+2, rdx <- C, rcx <- C
 30:     AssertAssignable:34(v2, Type: class 'num', '', instantiator_type_args(v0), function_type_args(v0)) T{Type: class 'num'?}
 32:     PushArgument(v2 T{Type: class 'num'?})
 33:     ParallelMove rax <- S-4
 34:     CheckNull:36(v9)
 36:     v11 <- StaticCall:38( <=<0> v9, v2 T{Type: class 'num'?}) T{bool}
 37:     ParallelMove rax <- rax
 38:     Branch if StrictCompare:42(===, v11, v6) goto (5, 6)
 40: B5[target]:46 ParallelMove rax <- C
 42:     PushArgument(v2 T{Type: class 'num'?})
 44:     PushArgument(v7 T{Type: class '_int64@0150898'})
 46:     v13 <- InstanceCall:54( %<0>, v2 T{Type: class 'num'?}, v7 T{Type: class '_int64@0150898'} IC[0: ])
 47:     ParallelMove rax <- rax, rcx <- C
 48:     Branch if CheckedSmiComparison:56(==, v13, v15) T{bool} goto (7, 8)
 50: B7[target]:64
 51:     ParallelMove rax <- C
 52:     Return:70(v21)
 54: B8[target]:66
 56:     ParallelMove rdx <- S-3
 56:     v32 <- UnboxInt64(v7 T{Type: class '_int64@0150898'}) [-9223372036854775808, 9223372036854775807] T{Type: class '_int64@0150898'}
 58:     ParallelMove rdx <- rdx
 58:     v19 <- BinaryInt64Op(+ [tr], v32 T{Type: class '_int64@0150898'}, v40) [-9223372036854775808, 9223372036854775807] T{Type: class '_int64@0150898'}
 60:     v34 <- BoxInt64(v19) [-9223372036854775808, 9223372036854775807] T{Type: class '_int64@0150898'}
 62:     ParallelMove rdx <- rax, rcx <- S+2, rax <- rcx goto:76 B9

The first problem:

• v2 (value of n) is checked for being num even though I would expect TFA to infer this information for us (more precisely it should infer that v2 is a non-nullable int). /cc @alexmarkov

This can be worked around in user code by changing type annotation on n from var n to int n. This brings around 15% performance improvement.

After this change the graph becomes this:

 26: B9[join]:72 pred(B4, B8) {
      v7 <- phi(v3, v42 T{Type: class '_int64@0150898'}) alive [-9223372036854775808, 9223372036854775807] T{Type: class '_int64@0150898'}
}
 27:     ParallelMove S-3 <- rbx
 28:     CheckStackOverflow:78(depth 1)
 30:     PushArgument(v7)
 32:     PushArgument(v7)
 34:     v9 <- StaticCall:32( *<0> v7, v7)
 35:     ParallelMove rax <- rax
 36:     CheckNull:34(v9)
 38:     ParallelMove rax <- rax
 38:     v40 <- UnboxInt64(v9 T{!null}) [-9223372036854775808, 9223372036854775807] T{Type: class '_int64@0150898'}
 39:     ParallelMove rcx <- S-4
 40:     Branch if RelationalOp(<=, v40, v36) T{bool} goto (5, 6)
 42: B5[target]:44 ParallelMove rax <- C
 44:     PushArgument(v2 T{Type: class 'int'})
 46:     PushArgument(v7 T{Type: class '_int64@0150898'})
 48:     v13 <- StaticCall:54( %<0> v2 T{Type: class 'int'}, v7 T{Type: class '_int64@0150898'}) T{Type: class 'num'?}
 49:     ParallelMove rax <- rax, rcx <- C
 50:     Branch if CheckedSmiComparison:56(==, v13, v15) T{bool} goto (7, 8)
 52: B7[target]:64
 53:     ParallelMove rax <- C
 54:     Return:70(v21)
 56: B8[target]:66
 58:     ParallelMove rdx <- S-3
 58:     v38 <- UnboxInt64(v7 T{Type: class '_int64@0150898'}) [-9223372036854775808, 9223372036854775807] T{Type: class '_int64@0150898'}
 60:     ParallelMove rdx <- rdx
 60:     v19 <- BinaryInt64Op(+ [tr], v38 T{Type: class '_int64@0150898'}, v50) [-9223372036854775808, 9223372036854775807] T{Type: class '_int64@0150898'}
 62:     v42 <- BoxInt64(v19) [-9223372036854775808, 9223372036854775807] T{Type: class '_int64@0150898'}
 64:     ParallelMove rbx <- rbx, rax <- S+2, rdx <- rcx, rcx <- S-4 goto:76 B9

Notice that

1. v9 <- StaticCall:32( *<0> v7, v7) is not inlined
2. v13 <- StaticCall:54( %<0> v2, v7 is not inlined
3. v7 phi is not unboxed across the loop.

These are three separate problems.

1. * is not inlined because in TryOptimizeStaticCallUsingStaticTypes we think that v7 can be nullable integer. (later we infer that it can't - but we don't rerun specialization code at later stages - which we should probably do).
2. % is not inlined because we don't have non-speculative implementation for it. I filed upstream bug for this AOT: implement non-speculative int64 versions of MOD and TRUNCDIV dart-lang/sdk#33967
3. v7 is not unboxed across the loop because the heuristic we have in UnboxPhi is too conservative.

/cc @aartbik this probably fits well into your overarching work on arithmetic so I am tentatively assigning this to you.

[aartbik]

On my desktop, with slightly larger request:

JIT vs AOT:
executed in 0:00:05.369510 (verify="500000th prime is 7368787")
executed in 0:00:17.198906 (verify="500000th prime is 7368787")

[aartbik]

I added this to our internally monitored micro benchmarks.

[aartbik]

Note: spin off issues that are still pending:

dart-lang/sdk#33967
dart-lang/sdk#34072