Package org.eclipse.collections.impl.parallel

Class ParallelIterate

  • public final class ParallelIterate
extends Object
    The ParallelIterate class contains several parallel algorithms that work with Collections. All the higher level parallel algorithms depend on the basic parallel algorithm named forEach. The forEach algorithm employs a batching fork and join approach.

    All Collections that are not either a RandomAccess or List are first converted to a Java array using Iterate.toArray(Iterable), and then run with one of the ParallelArrayIterate.forEach methods.

    See Also:
    ParallelArrayIterate

    • Method Detail

      • forEachWithIndex

        public static <T> void forEachWithIndex​(Iterable<T> iterable,
                                        ObjectIntProcedure<? super T> objectIntProcedure)
        Iterate over the collection specified, in parallel batches using default runtime parameter values. The ObjectIntProcedure used must be stateless, or use concurrent aware objects if they are to be shared.

        e.g. 

         Map<Integer, Object> chm = new ConcurrentHashMap<Integer, Object>();
 ParallelIterate.forEachWithIndex(collection, (each, index) -> chm.put(index, each));

      • forEachWithIndex

        public static <T,​BT extends ObjectIntProcedure<? super T>> void forEachWithIndex​(Iterable<T> iterable,
                                                                                       BT procedure,
                                                                                       Executor executor)
        Iterate over the collection specified in parallel batches using the default runtime parameters. The ObjectIntProcedure used must be stateless, or use concurrent aware objects if they are to be shared. The code is executed against the specified executor. 
        e.g.
 Map<Integer, Object> chm = new ConcurrentHashMap<Integer, Object>();
 ParallelIterate.forEachWithIndex(collection, (each, index) -> chm.put(index, each), executor);
        Parameters:
        executor - Use this executor for all execution.

      • forEachWithIndex

        public static <T,​BT extends ObjectIntProcedure<? super T>> void forEachWithIndex​(Iterable<T> iterable,
                                                                                       BT procedure,
                                                                                       int minForkSize,
                                                                                       int taskCount)
        Iterate over the collection specified in parallel batches. The ObjectIntProcedure used must be stateless, or use concurrent aware objects if they are to be shared. The specified minimum fork size and task count are used instead of the default values.
        Parameters:
        minForkSize - Only run in parallel if input collection is longer than this.
        taskCount - How many parallel tasks to submit to the executor.
        See Also:
        forEachWithIndex(Iterable, ObjectIntProcedure)

      • forEach

        public static <T> void forEach​(Iterable<T> iterable,
                               Procedure<? super T> procedure)
        Iterate over the collection specified in parallel batches using default runtime parameter values. The Procedure used must be stateless, or use concurrent aware objects if they are to be shared.

        e.g. 

         Map<Object, Boolean> chm = new ConcurrentHashMap<Object, Boolean>();
 ParallelIterate.forEach(collection, each -> chm.put(each, Boolean.TRUE));

      • forEach

        public static <T> void forEach​(Iterable<T> iterable,
                               Procedure<? super T> procedure,
                               int batchSize)
        Iterate over the collection specified in parallel batches using default runtime parameter values. The Procedure used must be stateless, or use concurrent aware objects if they are to be shared.

        e.g. 

         Map<Object, Boolean> chm = new ConcurrentHashMap<Object, Boolean>();
 ParallelIterate.forEachBatchSize(collection, each -> chm.put(each, Boolean.TRUE), 100);

      • forEach

        public static <T> void forEach​(Iterable<T> iterable,
                               Procedure<? super T> procedure,
                               int batchSize,
                               Executor executor)

      • forEach

        public static <T,​BT extends Procedure<? super T>> void forEach​(Iterable<T> iterable,
                                                                     BT procedure,
                                                                     Executor executor)
        Iterate over the collection specified in parallel batches using default runtime parameter values and the specified executor. The Procedure used must be stateless, or use concurrent aware objects if they are to be shared.
        Parameters:
        executor - Use this executor for all execution.
        See Also:
        forEach(Iterable, Procedure)

      • forEach

        public static <T,​BT extends Procedure<? super T>> void forEach​(Iterable<T> iterable,
                                                                     BT procedure,
                                                                     int minForkSize,
                                                                     int taskCount)
        Iterate over the collection specified in parallel batches using the specified minimum fork and task count sizes. The Procedure used must be stateless, or use concurrent aware objects if they are to be shared.
        Parameters:
        minForkSize - Only run in parallel if input collection is longer than this.
        taskCount - How many parallel tasks to submit to the executor. TODO: How does the taskCount relate to the number of threads in the executor?
        See Also:
        forEach(Iterable, Procedure)

      • forEach

        public static <T,​BT extends Procedure<? super T>> void forEach​(Iterable<T> iterable,
                                                                     BT procedure,
                                                                     int minForkSize,
                                                                     int taskCount,
                                                                     Executor executor)

      • forEach

        public static <T,​BT extends Procedure<? super T>> void forEach​(Iterable<T> iterable,
                                                                     ProcedureFactory<BT> procedureFactory,
                                                                     Combiner<BT> combiner,
                                                                     int batchSize)
        Iterate over the collection specified in parallel batches using the default values for the task size. The ProcedureFactory can create stateful closures that will be collected and combined using the specified Combiner. 
        e.g. The ParallelIterate.select() implementation

 CollectionCombiner<T, SelectProcedure<T>> combiner = CollectionCombiner.forSelect(collection);
 ParallelIterate.forEach(collection, new SelectProcedureFactory<T>(predicate, taskSize), combiner, 1000);

      • forEach

        public static <T,​BT extends Procedure<? super T>> void forEach​(Iterable<T> iterable,
                                                                     ProcedureFactory<BT> procedureFactory,
                                                                     Combiner<BT> combiner,
                                                                     int minForkSize,
                                                                     int taskCount)
        Iterate over the collection specified in parallel batches using the default values for the task size. The ProcedureFactory can create stateful closures that will be collected and combined using the specified Combiner. 
        e.g. The ParallelIterate.select() implementation

 int taskCount = Math.max(DEFAULT_PARALLEL_TASK_COUNT, collection.size() / DEFAULT_MIN_FORK_SIZE);
 final int taskSize = collection.size() / taskCount / 2;
 CollectionCombiner<T, SelectProcedure<T>> combiner = CollectionCombiner.forSelect(collection);
 ParallelIterate.forEach(collection, new SelectProcedureFactory<T>(predicate, taskSize), combiner, DEFAULT_MIN_FORK_SIZE, taskCount);

      • forEachInListOnExecutor

        public static <T,​BT extends Procedure<? super T>> void forEachInListOnExecutor​(List<T> list,
                                                                                     ProcedureFactory<BT> procedureFactory,
                                                                                     Combiner<BT> combiner,
                                                                                     int minForkSize,
                                                                                     int taskCount,
                                                                                     Executor executor)

      • select

        public static <T> Collection<T> select​(Iterable<T> iterable,
                                       Predicate<? super T> predicate,
                                       boolean allowReorderedResult)
        Same effect as Iterate.select(Iterable, Predicate), but executed in parallel batches, and with a potentially reordered result.
        Parameters:
        allowReorderedResult - If the result can be in a different order. Allowing reordering may yield faster execution.
        Returns:
        The selected elements. The Collection will be of the same type (List or Set) as the input.

      • select

        public static <T,​R extends Collection<T>> R select​(Iterable<T> iterable,
                                                         Predicate<? super T> predicate,
                                                         R target,
                                                         boolean allowReorderedResult)
        Same effect as Iterate.select(Iterable, Predicate), but executed in parallel batches, and writing output into the specified collection.
        Parameters:
        target - Where to write the output.
        allowReorderedResult - If the result can be in a different order. Allowing reordering may yield faster execution.
        Returns:
        The 'target' collection, with the selected elements added.

      • select

        public static <T,​R extends Collection<T>> R select​(Iterable<T> iterable,
                                                         Predicate<? super T> predicate,
                                                         R target,
                                                         int batchSize,
                                                         Executor executor,
                                                         boolean allowReorderedResult)
        Same effect as Iterate.select(Iterable, Predicate), but executed in parallel batches, and writing output into the specified collection.
        Parameters:
        target - Where to write the output.
        allowReorderedResult - If the result can be in a different order. Allowing reordering may yield faster execution.
        Returns:
        The 'target' collection, with the selected elements added.

      • reject

        public static <T> Collection<T> reject​(Iterable<T> iterable,
                                       Predicate<? super T> predicate,
                                       boolean allowReorderedResult)
        Same effect as Iterate.reject(Iterable, Predicate), but executed in parallel batches, and with a potentially reordered result.
        Parameters:
        allowReorderedResult - If the result can be in a different order. Allowing reordering may yield faster execution.
        Returns:
        The rejected elements. The Collection will be of the same type (List or Set) as the input.

      • reject

        public static <T,​R extends Collection<T>> R reject​(Iterable<T> iterable,
                                                         Predicate<? super T> predicate,
                                                         R target,
                                                         boolean allowReorderedResult)
        Same effect as Iterate.reject(Iterable, Predicate), but executed in parallel batches, and writing output into the specified collection.
        Parameters:
        target - Where to write the output.
        allowReorderedResult - If the result can be in a different order. Allowing reordering may yield faster execution.
        Returns:
        The 'target' collection, with the rejected elements added.

      • reject

        public static <T,​R extends Collection<T>> R reject​(Iterable<T> iterable,
                                                         Predicate<? super T> predicate,
                                                         R target,
                                                         int batchSize,
                                                         Executor executor,
                                                         boolean allowReorderedResult)

      • collect

        public static <T,​V> Collection<V> collect​(Iterable<T> iterable,
                                                Function<? super T,​V> function,
                                                boolean allowReorderedResult)
        Same effect as Iterate.collect(Iterable, Function), but executed in parallel batches, and with potentially reordered result.
        Parameters:
        allowReorderedResult - If the result can be in a different order. Allowing reordering may yield faster execution.
        Returns:
        The collected elements. The Collection will be of the same type (List or Set) as the input.

      • collect

        public static <T,​V,​R extends Collection<V>> R collect​(Iterable<T> iterable,
                                                                  Function<? super T,​V> function,
                                                                  R target,
                                                                  boolean allowReorderedResult)
        Same effect as Iterate.collect(Iterable, Function), but executed in parallel batches, and writing output into the specified collection.
        Parameters:
        target - Where to write the output.
        allowReorderedResult - If the result can be in a different order. Allowing reordering may yield faster execution.
        Returns:
        The 'target' collection, with the collected elements added.

      • collect

        public static <T,​V,​R extends Collection<V>> R collect​(Iterable<T> iterable,
                                                                  Function<? super T,​V> function,
                                                                  R target,
                                                                  int batchSize,
                                                                  Executor executor,
                                                                  boolean allowReorderedResult)

      • flatCollect

        public static <T,​V,​R extends Collection<V>> R flatCollect​(Iterable<T> iterable,
                                                                      Function<? super T,​Collection<V>> function,
                                                                      R target,
                                                                      boolean allowReorderedResult)

      • flatCollect

        public static <T,​V,​R extends Collection<V>> R flatCollect​(Iterable<T> iterable,
                                                                      Function<? super T,​Collection<V>> function,
                                                                      R target,
                                                                      int batchSize,
                                                                      Executor executor,
                                                                      boolean allowReorderedResult)

      • collectIf

        public static <T,​V> Collection<V> collectIf​(Iterable<T> iterable,
                                                  Predicate<? super T> predicate,
                                                  Function<? super T,​V> function,
                                                  boolean allowReorderedResult)
        Same effect as Iterate.collectIf(Iterable, Predicate, Function), but executed in parallel batches, and with potentially reordered results.
        Parameters:
        allowReorderedResult - If the result can be in a different order. Allowing reordering may yield faster execution.
        Returns:
        The collected elements. The Collection will be of the same type as the input (List or Set)

      • collectIf

        public static <T,​V,​R extends Collection<V>> R collectIf​(Iterable<T> iterable,
                                                                    Predicate<? super T> predicate,
                                                                    Function<? super T,​V> function,
                                                                    R target,
                                                                    boolean allowReorderedResult)
        Same effect as Iterate.collectIf(Iterable, Predicate, Function), but executed in parallel batches, and writing output into the specified collection.
        Parameters:
        target - Where to write the output.
        allowReorderedResult - If the result can be in a different order. Allowing reordering may yield faster execution.
        Returns:
        The 'target' collection, with the collected elements added.

      • collectIf

        public static <T,​V,​R extends Collection<V>> R collectIf​(Iterable<T> iterable,
                                                                    Predicate<? super T> predicate,
                                                                    Function<? super T,​V> function,
                                                                    R target,
                                                                    int batchSize,
                                                                    Executor executor,
                                                                    boolean allowReorderedResult)

      • aggregateBy

        public static <T,​K,​V> MutableMap<K,​V> aggregateBy​(Iterable<T> iterable,
                                                                    Function<? super T,​? extends K> groupBy,
                                                                    Function0<? extends V> zeroValueFactory,
                                                                    Function2<? super V,​? super T,​? extends V> nonMutatingAggregator)

      • aggregateBy

        public static <T,​K,​V,​R extends MutableMapIterable<K,​V>> R aggregateBy​(Iterable<T> iterable,
                                                                                              Function<? super T,​? extends K> groupBy,
                                                                                              Function0<? extends V> zeroValueFactory,
                                                                                              Function2<? super V,​? super T,​? extends V> nonMutatingAggregator,
                                                                                              R mutableMap)

      • aggregateBy

        public static <T,​K,​V> MutableMap<K,​V> aggregateBy​(Iterable<T> iterable,
                                                                    Function<? super T,​? extends K> groupBy,
                                                                    Function0<? extends V> zeroValueFactory,
                                                                    Function2<? super V,​? super T,​? extends V> nonMutatingAggregator,
                                                                    int batchSize)

      • aggregateBy

        public static <T,​K,​V,​R extends MutableMapIterable<K,​V>> R aggregateBy​(Iterable<T> iterable,
                                                                                              Function<? super T,​? extends K> groupBy,
                                                                                              Function0<? extends V> zeroValueFactory,
                                                                                              Function2<? super V,​? super T,​? extends V> nonMutatingAggregator,
                                                                                              R mutableMap,
                                                                                              int batchSize)

      • aggregateBy

        public static <T,​K,​V> MutableMap<K,​V> aggregateBy​(Iterable<T> iterable,
                                                                    Function<? super T,​? extends K> groupBy,
                                                                    Function0<? extends V> zeroValueFactory,
                                                                    Function2<? super V,​? super T,​? extends V> nonMutatingAggregator,
                                                                    int batchSize,
                                                                    Executor executor)

      • aggregateBy

        public static <T,​K,​V,​R extends MutableMapIterable<K,​V>> R aggregateBy​(Iterable<T> iterable,
                                                                                              Function<? super T,​? extends K> groupBy,
                                                                                              Function0<? extends V> zeroValueFactory,
                                                                                              Function2<? super V,​? super T,​? extends V> nonMutatingAggregator,
                                                                                              R mutableMap,
                                                                                              int batchSize,
                                                                                              Executor executor)

      • aggregateInPlaceBy

        public static <T,​K,​V> MutableMap<K,​V> aggregateInPlaceBy​(Iterable<T> iterable,
                                                                           Function<? super T,​? extends K> groupBy,
                                                                           Function0<? extends V> zeroValueFactory,
                                                                           Procedure2<? super V,​? super T> mutatingAggregator)

      • aggregateInPlaceBy

        public static <T,​K,​V,​R extends MutableMapIterable<K,​V>> R aggregateInPlaceBy​(Iterable<T> iterable,
                                                                                                     Function<? super T,​? extends K> groupBy,
                                                                                                     Function0<? extends V> zeroValueFactory,
                                                                                                     Procedure2<? super V,​? super T> mutatingAggregator,
                                                                                                     R mutableMap)

      • aggregateInPlaceBy

        public static <T,​K,​V> MutableMap<K,​V> aggregateInPlaceBy​(Iterable<T> iterable,
                                                                           Function<? super T,​? extends K> groupBy,
                                                                           Function0<? extends V> zeroValueFactory,
                                                                           Procedure2<? super V,​? super T> mutatingAggregator,
                                                                           int batchSize)

      • aggregateInPlaceBy

        public static <T,​K,​V,​R extends MutableMapIterable<K,​V>> R aggregateInPlaceBy​(Iterable<T> iterable,
                                                                                                     Function<? super T,​? extends K> groupBy,
                                                                                                     Function0<? extends V> zeroValueFactory,
                                                                                                     Procedure2<? super V,​? super T> mutatingAggregator,
                                                                                                     R mutableMap,
                                                                                                     int batchSize)

      • aggregateInPlaceBy

        public static <T,​K,​V> MutableMap<K,​V> aggregateInPlaceBy​(Iterable<T> iterable,
                                                                           Function<? super T,​? extends K> groupBy,
                                                                           Function0<? extends V> zeroValueFactory,
                                                                           Procedure2<? super V,​? super T> mutatingAggregator,
                                                                           int batchSize,
                                                                           Executor executor)

      • aggregateInPlaceBy

        public static <T,​K,​V,​R extends MutableMapIterable<K,​V>> R aggregateInPlaceBy​(Iterable<T> iterable,
                                                                                                     Function<? super T,​? extends K> groupBy,
                                                                                                     Function0<? extends V> zeroValueFactory,
                                                                                                     Procedure2<? super V,​? super T> mutatingAggregator,
                                                                                                     R mutableMap,
                                                                                                     int batchSize,
                                                                                                     Executor executor)

      • newPooledExecutor

        public static ExecutorService newPooledExecutor​(int newPoolSize,
                                                String poolName,
                                                boolean useDaemonThreads)
        Returns a brand new ExecutorService using the specified poolName with the specified maximum thread pool size. The same poolName may be used more than once resulting in multiple pools with the same name.

        The pool will be initialised with newPoolSize threads. If that number of threads are in use and another thread is requested, the pool will reject execution and the submitting thread will execute the task.

      • newPooledExecutor

        public static ExecutorService newPooledExecutor​(String poolName,
                                                boolean useDaemonThreads)
        Returns a brand new ExecutorService using the specified poolName and uses the optional property named to set the maximum thread pool size. The same poolName may be used more than once resulting in multiple pools with the same name.

      • getDefaultTaskCount

        public static int getDefaultTaskCount()

      • getDefaultMaxThreadPoolSize

        public static int getDefaultMaxThreadPoolSize()

      • getTaskRatio

        public static int getTaskRatio()