.. include:: header.txt

===============
 Process Pools
===============

The `processing.pool` module has one public class:

    **class** `Pool(processes=None)`
        A class representing a pool of worker processes.  

        Tasks can be offloaded to the pool and the results dealt with
        when they become available.


Pool objects
============

`Pool` has the following public methods:

    `__init__(processes=None)`
        The constructor creates and starts `processes` worker
        processes.  If `processes` is `None` then `cpuCount()` is used
        to find a default or 1 if `cpuCount()` raises `NotImplemented`.

    `apply(func, args=(), kwds={})`
        Equivalent of the `apply()` builtin function.  It blocks till
        the result is ready.

    `apply_async(func, args=(), kwds={}, callback=None)`
        A variant of the `apply()` method which returns a
        result object --- see `Asynchronous result objects`_.

        If `callback` is specified then it should be a callable which
        accepts a single argument.  When the result becomes ready
        `callback` is applied to it (unless the call failed).

    `map(func, iterable, chunksize=None)`
        A parallel equivalent of the `map()` builtin function.  It
        blocks till the result is ready.  

        This method chops the iterable into a number of chunks which
        it submits to the process pool as separate tasks.  The
        (maximum) size of these chunks can be specified by setting
        `chunksize` to a positive integer.

    `map_async(func, iterable, chunksize=None, callback=None)`
        A variant of the `map()` method which returns a result object
        --- see `Asynchronous result objects`_.

        If `callback` is specified then it should be a callable which
        accepts a single argument.  When the result becomes ready
        `callback` is applied to it (unless the call failed).
        
    `imap(func, iterable)`
        An equivalent of `itertools.imap()`.

        Note that this treats each element of the iterable as a
        separate task, so for a long iterable where the function is
        cheap to evaluate this likely to be **much** slower than using
        `map()`.

        Also notice that the `next(timeout=None)` method of the
        iterator returned by the `imap()` method has a timeout
        parameter.  `next(timeout)` will raise
        `processing.TimeoutError` if the result cannot be returned
        with `timeout` seconds.

    `imap_unordered(func, iterable)`
        The same as `imap()` except that the ordering of the results
        from the returned iterator should be considered arbitrary.
        (Only when there is only one worker process is the order
        guaranteed to be "correct".)

    `shutdown()`
        Tells the worker processes to shutdown.  Gets called
        automatically when the pool object is garbage collected or
        when the process shuts down.

    `join()`
        Tells the worker processes to shutdown and waits for them to
        finish.


Asynchronous result objects
===========================

The result objects returns by `apply_async()` and `map_async()` have
the following public methods:

    `get(timeout=None)`
        Returns the result when it arrives.  If `timeout` is not
        `None` and the result does not arrive within `timeout` seconds
        then `processing.TimeoutError` is raised.  If the remote call
        raised an exception then that exception will be reraised by `get()`.

    `wait(timeout=None)`
        Waits until the result is available or until `timeout` seconds
        pass.

    `ready()`
        Returns whether the call has completed.

    `successful()`
        Returns whether the call completed without raising an
        exception.  Will raise `AssertionError` if the result is not
        ready.


Examples
========

The following example demonstrates the use of a pool::

    from processing import Pool

    def f(x):
        return x*x

    if __name__ == '__main__':
        pool = Pool(processes=4)              # start 4 worker processes

        result = pool.apply_async(f, [10])    # evaluate "f(10)" asynchronously
        print result.get(timeout=1)           # prints "100" unless your computer is *very* slow

        print pool.map(f, range(10))          # prints "[0, 1, 4,..., 81]"

        it = pool.imap(f, range(10))
        print it.next()                       # prints "0"
        print it.next()                       # prints "1"
        print it.next(timeout=1)              # prints "4" unless your computer is *very* slow

        import time
        result = pool.apply_async(time.sleep, [10])
        print result.get(timeout=1)           # raises `TimeoutError`
                    


See also `test_pool.py <../test/test_pool.py>`_.

.. _Prev: proxy-objects.html
.. _Up: processing-ref.html
.. _Next: connection-ref.html
