Skip to content

Bug of mapResult in ParIterableLike #104

Closed
@dengziming

Description

@dengziming

here is my code:

object App {

  val executor: ThreadPoolExecutor = Executors.newFixedThreadPool(1, threadFactory()).asInstanceOf[ThreadPoolExecutor]

  val executionContext: ExecutionContextExecutor = ExecutionContext.fromExecutor(executor)
  
  /**
    * Create a thread factory ,sets the threads to daemon.
    */
  def threadFactory(): ThreadFactory = {

    runnable: Runnable => {
      val backingThreadFactory = Executors.defaultThreadFactory
      val thread = backingThreadFactory.newThread(runnable)
      thread.setDaemon(true)
      thread
    }
  }
  
  def main(args: Array[String]): Unit = {

    val par = Array(1).par
    par.tasksupport = new ExecutionContextTaskSupport(executionContext)
    // par.foreach(println(_))
    par.flatMap(i => Array(1)).foreach(println)
  }
}

here is the screenshot from jconsole

  1. the thread I created in my code submit a Task to a ThreadPool and return a future
    image

  2. the thread in ThreadPool submit a Task again to the ThreadPool, since the ThreadPool size=1, so it will block infinitely.

image

I inspect the code of scala.connection.parallel.ParIterableLike.flatMap and find this procedure is troublesome: new FlatMap mapResult { _.resultWithTaskSupport }.
I think this is creating 2 tasks like a tree, with ResultMapping as leaf Node and FlatMap as root Node, first we submit FlatMap to ThreadPool which depends on ResultMapping, so we submit ResultMapping to the ThreadPool but there are no resources in ThreadPool.

So I don't think it's necessary to add mapResult as a leaf node, the code in which can just be moved out.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions