NonBlockingThreadPool

public class NonBlockingThreadPool<Environment> : ComputeThreadPool where Environment : ConcurrencyPlatform

An efficient, work-stealing, general purpose compute thread pool.

NonBlockingThreadPool can be cleaned up by calling shutDown() which will block until all threads in the threadpool have exited. If shutDown() is never called, NonBlockingThreadPool will never be deallocated.

NonBlockingThreadPool uses atomics to implement a non-blocking thread pool. During normal execution, no locks are acquired or released. This can result in efficient parallelism across many cores. NonBlockingThreadPool is designed to scale from laptops to high-core-count servers. Although the thread pool size can be manually tuned, often the most efficient configuration is a one-to-one mapping between hardware threads and worker threads, as this allows full use of the hardware while avoiding unnecessary context switches. I/O heavy workloads may want to reduce the thread pool count to dedicate a core or two to I/O processing.

Each thread managed by this thread pool maintains its own fixed-size pending task queue. The workers loop, trying to get their next tasks from their own queue first, and if that queue is empty, the worker tries to steal work from the pending task queues of other threads in the pool.

NonBlockingThreadPool implements important optimizations based on the calling thread. There are key fast-paths taken when calling functions on NonBlockingThreadPool from threads that have been registered with the pool (or from threads managed by the pool itself). In order to help users build performant applications, NonBlockingThreadPool will trap (and exit the process) if functions are called on it from non-fast-path’d threads by default. You can change this behavior by setting allowNonFastPathThreads: true at initialization.

In order to avoid wasting excessive CPU cycles, the worker threads managed by NonBlockingThreadPool will suspend themselves (using locks to inform the host kernel). NonBlockingThreadPool is parameterized by an environment, which allows this thread pool to seamlessly interoperate within a larger application by reusing its concurrency primitives (such as locks and condition variables, which are used for thread parking), as well as even allowing a custom thread allocator.

Local tasks typically execute in LIFO order, which is often optimal for cache locality of compute intensive tasks. Other threads attempt to steal work “FIFO”-style, which admits an efficient (dynamic) schedule for typical divide-and-conquor algorithms.

This implementation is inspired by the Eigen thread pool library, TFRT, as well as:

"Thread Scheduling for Multiprogrammed Multiprocessors"
Nimar S. Arora, Robert D. Blumofe, C. Greg Plaxton
Show on GitHub

  • Undocumented

    Declaration

    Swift

    public typealias Task = () -> Void
    Show on GitHub

  • Undocumented

    Declaration

    Swift

    public typealias ThrowingTask = () throws -> Void
    Show on GitHub

  • Initialize a new thread pool with threadCount threads using threading environment environment.

    Declaration

    Swift

    public init(
  name: String,
  threadCount: Int,
  environment: Environment,
  externalFastPathThreadCount: Int = 1,
  allowNonFastPathThreads: Bool = false
)

    Parameters

    name

    a human-readable name for the threadpool.
    threadCount

    the number of worker threads in the thread pool.
    environment

    an instance of the environment.
    externalFastPathThreadCount

    the maximum number of external threads with fast-path access to the threadpool.
    allowNonFastPathThreads

    true if non-fast-path’d threads are allowed to submit work into the pool or not. (Note: non-fast-path’d threads can always dispatch work into the pool.)
    Show on GitHub

  • Registers the current thread with the thread pool for fast-path operation.

    Declaration

    Swift

    public func registerCurrentThread()
    Show on GitHub

  • Declaration

    Swift

    public func dispatch(_ fn: @escaping Task)
    Show on GitHub

  • Declaration

    Swift

    public func join(_ a: Task, _ b: Task)
    Show on GitHub

  • Declaration

    Swift

    public func join(_ a: ThrowingTask, _ b: ThrowingTask) throws
    Show on GitHub

  • Executes fn, optionally in parallel, spanning the range 0..<n.

    Declaration

    Swift

    public func parallelFor(n: Int, _ fn: VectorizedParallelForBody)
    Show on GitHub

  • Executes fn, optionally in parallel, spanning the range 0..<n.

    Declaration

    Swift

    public func parallelFor(n: Int, _ fn: ThrowingVectorizedParallelForBody) throws
    Show on GitHub

  • Requests that all threads in the threadpool exit and cleans up their associated resources.

    This function returns only once all threads have exited and their resources have been deallocated.

    Note: if a work item was submitted to the threadpool that never completes (i.e. has an infinite loop), this function will never return.

    Declaration

    Swift

    public func shutDown()
    Show on GitHub

  • Declaration

    Swift

    public var maxParallelism: Int { get }
    Show on GitHub

  • Declaration

    Swift

    public var currentThreadIndex: Int? { get }
    Show on GitHub

Available where Environment: DefaultInitializable

  • Creates self using a default-initialized Environment, and the specified name and threadCount.

    Declaration

    Swift

    public convenience init(name: String, threadCount: Int)
    Show on GitHub