java.lang.IllegalArgumentException: Self-suppression not permitted

[ilya-epifanov]

This is a random error message I've got. I understand that the root cause was an OOM which is caused by my program, not the scala library, but it still might be that scala.collection.parallel.Task.mergeThrowables doesn't use Throwable.addSuppressed properly.

java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: Self-suppression not permitted
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at java.util.concurrent.ForkJoinTask.getThrowableException(ForkJoinTask.java:593)
	at java.util.concurrent.ForkJoinTask.reportException(ForkJoinTask.java:677)
	at java.util.concurrent.ForkJoinTask.join(ForkJoinTask.java:720)
	at scala.collection.parallel.ForkJoinTasks$WrappedTask.sync(Tasks.scala:375)
	at scala.collection.parallel.ForkJoinTasks$WrappedTask.sync$(Tasks.scala:375)
	at scala.collection.parallel.AdaptiveWorkStealingForkJoinTasks$WrappedTask.sync(Tasks.scala:440)
	at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask.internal(Tasks.scala:170)
	at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask.internal$(Tasks.scala:153)
	at scala.collection.parallel.AdaptiveWorkStealingForkJoinTasks$WrappedTask.internal(Tasks.scala:440)
	at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask.compute(Tasks.scala:146)
	at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask.compute$(Tasks.scala:145)
	at scala.collection.parallel.AdaptiveWorkStealingForkJoinTasks$WrappedTask.compute(Tasks.scala:440)
	at java.util.concurrent.RecursiveAction.exec(RecursiveAction.java:189)
	at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
	at java.util.concurrent.ForkJoinTask.doJoin(ForkJoinTask.java:389)
	at java.util.concurrent.ForkJoinTask.join(ForkJoinTask.java:719)
	at scala.collection.parallel.ForkJoinTasks$WrappedTask.sync(Tasks.scala:375)
	at scala.collection.parallel.ForkJoinTasks$WrappedTask.sync$(Tasks.scala:375)
	at scala.collection.parallel.AdaptiveWorkStealingForkJoinTasks$WrappedTask.sync(Tasks.scala:440)
	at scala.collection.parallel.ForkJoinTasks.executeAndWaitResult(Tasks.scala:423)
	at scala.collection.parallel.ForkJoinTasks.executeAndWaitResult$(Tasks.scala:414)
	at scala.collection.parallel.ForkJoinTaskSupport.executeAndWaitResult(TaskSupport.scala:56)
	at scala.collection.parallel.ExecutionContextTasks.executeAndWaitResult(Tasks.scala:555)
	at scala.collection.parallel.ExecutionContextTasks.executeAndWaitResult$(Tasks.scala:555)
	at scala.collection.parallel.ExecutionContextTaskSupport.executeAndWaitResult(TaskSupport.scala:80)
	at scala.collection.parallel.ParIterableLike$ResultMapping.leaf(ParIterableLike.scala:956)
	at scala.collection.parallel.Task.$anonfun$tryLeaf$1(Tasks.scala:49)
	at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:12)
	at scala.util.control.Breaks$$anon$1.catchBreak(Breaks.scala:63)
	at scala.collection.parallel.Task.tryLeaf(Tasks.scala:52)
	at scala.collection.parallel.Task.tryLeaf$(Tasks.scala:46)
	at scala.collection.parallel.ParIterableLike$ResultMapping.tryLeaf(ParIterableLike.scala:951)
	at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask.compute(Tasks.scala:149)
	at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask.compute$(Tasks.scala:145)
	at scala.collection.parallel.AdaptiveWorkStealingForkJoinTasks$WrappedTask.compute(Tasks.scala:440)
	at java.util.concurrent.RecursiveAction.exec(RecursiveAction.java:189)
	at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
	at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
	at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
	at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
Caused by: java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: Self-suppression not permitted
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at java.util.concurrent.ForkJoinTask.getThrowableException(ForkJoinTask.java:593)
	at java.util.concurrent.ForkJoinTask.reportException(ForkJoinTask.java:677)
	at java.util.concurrent.ForkJoinTask.join(ForkJoinTask.java:720)
	at scala.collection.parallel.ForkJoinTasks$WrappedTask.sync(Tasks.scala:375)
	at scala.collection.parallel.ForkJoinTasks$WrappedTask.sync$(Tasks.scala:375)
	at scala.collection.parallel.AdaptiveWorkStealingForkJoinTasks$WrappedTask.sync(Tasks.scala:440)
	at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask.internal(Tasks.scala:170)
	at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask.internal$(Tasks.scala:153)
	at scala.collection.parallel.AdaptiveWorkStealingForkJoinTasks$WrappedTask.internal(Tasks.scala:440)
	at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask.compute(Tasks.scala:146)
	... 7 common frames omitted
Caused by: java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: Self-suppression not permitted
	... 21 common frames omitted
Caused by: java.lang.IllegalArgumentException: Self-suppression not permitted
	at java.lang.Throwable.addSuppressed(Throwable.java:1043)
	at scala.collection.parallel.Task.mergeThrowables(Tasks.scala:70)
	at scala.collection.parallel.Task.mergeThrowables$(Tasks.scala:68)
	at scala.collection.parallel.ParIterableLike$FlatMap.mergeThrowables(ParIterableLike.scala:1066)
	at scala.collection.parallel.Task.tryMerge(Tasks.scala:65)
	at scala.collection.parallel.Task.tryMerge$(Tasks.scala:62)
	at scala.collection.parallel.ParIterableLike$FlatMap.tryMerge(ParIterableLike.scala:1066)
	at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask.internal(Tasks.scala:173)
	at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask.internal$(Tasks.scala:153)
	at scala.collection.parallel.AdaptiveWorkStealingForkJoinTasks$WrappedTask.internal(Tasks.scala:440)
	at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask.compute(Tasks.scala:146)
	at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask.compute$(Tasks.scala:145)
	at scala.collection.parallel.AdaptiveWorkStealingForkJoinTasks$WrappedTask.compute(Tasks.scala:440)
	at java.util.concurrent.RecursiveAction.exec(RecursiveAction.java:189)
	at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
	at java.util.concurrent.ForkJoinPool.helpStealer(ForkJoinPool.java:1958)
	at java.util.concurrent.ForkJoinPool.awaitJoin(ForkJoinPool.java:2047)
	at java.util.concurrent.ForkJoinTask.doJoin(ForkJoinTask.java:390)
	at java.util.concurrent.ForkJoinTask.join(ForkJoinTask.java:719)
	... 14 common frames omitted
Caused by: java.lang.OutOfMemoryError: Java heap space

[aturner-tvpage]

I just saw this error again today myself, but without to OOM exception

[oxbowlakes]

We've just had a production process fall over with this one :-( It's made more complicated because the JVM is free to decide to do this:

The compiler in the server VM now provides correct stack backtraces for all "cold" built-in exceptions. For performance purposes, when such an exception is thrown a few times, the method may be recompiled. After recompilation, the compiler may choose a faster tactic using preallocated exceptions that do not provide a stack trace. To disable completely the use of preallocated exceptions, use this new flag: -XX:-OmitStackTraceInFastThrow.

This meant (I think) that different threads in a parallel collection task, notionally throwing different ClassCastExceptions had these changed by HotSpot into the same ClassCastException which then could not be merged by the scala collections library due to this bug. So for us, a workaround to the bug was adding the flag -XX:-OmitStackTraceInFastThrow

But - this feels like the proper fix is an easy one: change (in scala.collection.parallel.Tasks.scala)

private[parallel] def mergeThrowables(that: Task[_, _]) {
   if (this.throwable != null && that.throwable != null)
     this.throwable.addSuppressed(that.throwable)
   else if (this.throwable == null && that.throwable != null)
     this.throwable = that.throwable
}

To

private[parallel] def mergeThrowables(that: Task[_, _]) {
   if (this.throwable != null && that.throwable != null && this.throwable != that.throwable)
     this.throwable.addSuppressed(that.throwable)
   else if (this.throwable == null && that.throwable != null)
     this.throwable = that.throwable
}

[ritschwumm]

still alive and kicking - had this right now in scala 3.1.0

[joroKr21]

The suggested fix seems reasonable to me. But it looks like no one cared enough to submit a fix. These days the parallel collections live in a separate module and still use the old definition of mergeThrowables: https://github.com/scala/scala-parallel-collections/blob/main/core/src/main/scala/scala/collection/parallel/Tasks.scala#L71-L76

@ritschwumm care to submit a fix? I think this is also a problem if a user throws a cached exception no? So it should be possible to also reproduce it deterministically.

[joroKr21]

One questionable part of the fix is whether to use this.throwable != that.throwable or this.throwable ne that.throwable - since user-defined exceptions can have their own equals semantics, perhaps the latter would be more conservative.

[som-snytt]

I also noticed the bump, agreed with the proposal, and consulted my local repo of par-coll. I'll PR with reference ne, that's a good tweak.

Also wanted to point out the nice output from REPL:

scala> val t = new Throwable("oops")
val t: Throwable = java.lang.Throwable: oops

scala> t.addSuppressed(t)
java.lang.IllegalArgumentException: Self-suppression not permitted
  at java.base/java.lang.Throwable.addSuppressed(Throwable.java:1072)
  ... 32 elided
Caused by: java.lang.Throwable: oops
  ... 6 elided and 26 more

I remember @NthPortal contributed a fix for it, which seems a long time ago because it was the beginning of the pandemic, when we thought "this month there is a pandemic."

6 elided and 26 more sounds like a children's counting rhyme from the nursery, but I'm not actually sure any more what is counted.

Self-suppression not permitted would make a great protest t-shirt.

[ritschwumm]

sorry @joroKr21, don't understand the issue well enough to attempt fixing it. and of course i should have noticed this is not the right repo any more - having to explicitly add a dependency to my build.sbt might have told me...

[som-snytt]

That's OK ritschwumm, I needed the refresher.

scala/scala-parallel-collections#205

I'll close this here, although I think it could be migrated to the other repo.

[NthPortal]

@som-snytt I have no memory of working on this. I think you might be mixing it up either with something I've done on Using, or with scala/scala#8635

[som-snytt]

@NthPortal on REPL stack trace rendering scala/scala#8888