List of islands#

Islands implemented in Python#

class pygmo.mp_island(use_pool=True)#

Multiprocessing island.

New in version 2.10: The use_pool parameter (in previous versions, mp_island always used a process pool).

This user-defined island (UDI) will dispatch evolution tasks to an external Python process using the facilities provided by the standard Python multiprocessing module.

If the construction argument use_pool is True, then a process from a global pool shared between different instances of mp_island will be used. The pool is created either implicitly by the construction of the first mp_island object or explicitly via the init_pool() static method. The default number of processes in the pool is equal to the number of logical CPUs on the current machine. The pool’s size can be queried via get_pool_size(), and changed via resize_pool(). The pool can be stopped via shutdown_pool().

If use_pool is False, each evolution launched by an mp_island will be offloaded to a new process which will then be terminated at the end of the evolution.

Generally speaking, a process pool will be faster (and will use fewer resources) than spawning a new process for every evolution. A process pool, however, by its very nature limits the number of evolutions that can be run simultaneously on the system, and it introduces a serializing behaviour that might not be desirable in certain situations (e.g., when studying parallel evolution with migration in an archipelago).

Note

Due to certain implementation details of CPython, it is not possible to initialise, resize or shutdown the pool from a thread different from the main one. Normally this is not a problem, but, for instance, if the first mp_island instance is created in a thread different from the main one, an error will be raised. In such a situation, the user should ensure to call init_pool() from the main thread before spawning the secondary thread.

Warning

Due to internal limitations of CPython, sending an interrupt signal (e.g., by pressing Ctrl+C in an interactive Python session) while an mp_island is evolving might end up sending an interrupt signal also to the external evolution process(es). This can lead to unpredictable runtime behaviour (e.g., the session may hang). Although pygmo tries hard to limit as much as possible the chances of this occurrence, it cannot eliminate them completely. Users are thus advised to tread carefully with interrupt signals (especially in interactive sessions) when using mp_island.

Warning

Due to an upstream bug, when using Python 3.8 the multiprocessing machinery may lead to a hangup when exiting a Python session. As a workaround until the bug is resolved, users are advised to explicitly call shutdown_pool() before exiting a Python session.

Parameters

use_pool (bool) – if True, a process from a global pool will be used to run the evolution, otherwise a new process will be spawned for each evolution

Raises
get_extra_info()#

Island’s extra info.

If the island uses a process pool and the pool was previously shut down via shutdown_pool(), invoking this function will trigger the creation of a new pool.

Returns

a string containing information about the state of the island (e.g., number of processes in the pool, ID of the evolution process, etc.)

Return type

str

Raises

unspecified – any exception thrown by get_pool_size()

get_name()#

Island’s name.

Returns

"Multiprocessing island"

Return type

str

static get_pool_size()#

Get the size of the process pool.

If the process pool was previously shut down via shutdown_pool(), invoking this function will trigger the creation of a new pool.

Returns

the current size of the pool

Return type

int

Raises

unspecified – any exception thrown by init_pool()

static init_pool(processes=None)#

Initialise the process pool.

This method will initialise the process pool backing mp_island, if the pool has not been initialised yet or if the pool was previously shut down via shutdown_pool(). Otherwise, this method will have no effects.

Parameters

processes (None or an int) – the size of the pool (if None, the size of the pool will be equal to the number of logical CPUs on the system)

Raises

  • ValueError – if the pool does not exist yet and the function is being called from a thread different from the main one, or if processes is a non-positive value

  • TypeError – if processes is not None and not an int

property pid#

ID of the evolution process (read-only).

This property is available only if the island is not using a process pool.

Returns

the ID of the process running the current evolution, or None if no evolution is ongoing

Return type

int

Raises

ValueError – if the island is using a process pool

static resize_pool(processes)#

Resize pool.

This method will resize the process pool backing mp_island.

If the process pool was previously shut down via shutdown_pool(), invoking this function will trigger the creation of a new pool.

Parameters

processes (int) – the desired number of processes in the pool

Raises

  • TypeError – if the processes argument is not an int

  • ValueError – if the processes argument is not strictly positive

  • unspecified – any exception thrown by init_pool()

run_evolve(algo, pop)#

Evolve population.

This method will evolve the input population pop using the input algorithm algo, and return algo and the evolved population. The evolution is run either on one of the processes of the pool backing mp_island, or in a new separate process. If this island is using a pool, and the pool was previously shut down via shutdown_pool(), an exception will be raised.

Parameters
Returns

a tuple of 2 elements containing algo (i.e., the algorithm object that was used for the evolution) and the evolved population

Return type

tuple

Raises

  • RuntimeError – if the pool was manually shut down via shutdown_pool()

  • unspecified – any exception thrown by the evolution, by the (de)serialization of the input arguments or of the return value, or by the public interface of the process pool

static shutdown_pool()#

Shutdown pool.

New in version 2.8.

This method will shut down the process pool backing mp_island, after all pending tasks in the pool have completed.

After the process pool has been shut down, attempting to run an evolution on the island will raise an error. A new process pool can be created via an explicit call to init_pool() or one of the methods of the public API of mp_island which trigger the creation of a new process pool.

property use_pool#

Pool usage flag (read-only).

Returns

True if this island uses a process pool, False otherwise

Return type

bool

class pygmo.ipyparallel_island#

Ipyparallel island.

This user-defined island (UDI) will dispatch evolution tasks to an ipyparallel cluster. The communication with the cluster is managed via an ipyparallel.LoadBalancedView instance which is created either implicitly when the first evolution is run, or explicitly via the init_view() method. The LoadBalancedView instance is a global object shared among all the ipyparallel islands.

See also

https://ipyparallel.readthedocs.io/en/latest/

get_extra_info()#

Island’s extra info.

Returns

a string with extra information about the status of the island

Return type

str

get_name()#

Island’s name.

Returns

"Ipyparallel island"

Return type

str

static init_view(client_args=[], client_kwargs={}, view_args=[], view_kwargs={})#

Init the ipyparallel view.

New in version 2.12.

This method will initialise the ipyparallel.LoadBalancedView which is used by all ipyparallel islands to submit the evolution tasks to an ipyparallel cluster. If the ipyparallel.LoadBalancedView has already been created, this method will perform no action.

The input arguments client_args and client_kwargs are forwarded as positional and keyword arguments to the construction of an ipyparallel.Client instance. From the constructed client, an ipyparallel.LoadBalancedView instance is then created via the ipyparallel.Client.load_balanced_view() method, to which the positional and keyword arguments view_args and view_kwargs are passed.

Note that usually it is not necessary to explicitly invoke this method: an ipyparallel.LoadBalancedView is automatically constructed with default settings the first time an evolution task is submitted to an ipyparallel island. This method should be used only if it is necessary to pass custom arguments to the construction of the ipyparallel.Client or ipyparallel.LoadBalancedView objects.

Parameters

  • client_args (list) – the positional arguments used for the construction of the client

  • client_kwargs (dict) – the keyword arguments used for the construction of the client

  • view_args (list) – the positional arguments used for the construction of the view

  • view_kwargs (dict) – the keyword arguments used for the construction of the view

Raises

unspecified – any exception thrown by the constructor of ipyparallel.Client or by the ipyparallel.Client.load_balanced_view() method

run_evolve(algo, pop)#

Evolve population.

This method will evolve the input population pop using the input algorithm algo, and return algo and the evolved population. The evolution task is submitted to the ipyparallel cluster via a global ipyparallel.LoadBalancedView instance initialised either implicitly by the first invocation of this method, or by an explicit call to the init_view() method.

Parameters
Returns

a tuple of 2 elements containing algo (i.e., the algorithm object that was used for the evolution) and the evolved population

Return type

tuple

Raises

unspecified – any exception thrown by the evolution, by the creation of a ipyparallel.LoadBalancedView, or by the sumission of the evolution task to the ipyparallel cluster

static shutdown_view()#

Destroy the ipyparallel view.

New in version 2.12.

This method will destroy the ipyparallel.LoadBalancedView currently being used by the ipyparallel islands for submitting evolution tasks to an ipyparallel cluster. The view can be re-inited implicitly by submitting a new evolution task, or by invoking the init_view() method.

Islands exposed from C++#

class pygmo.thread_island(use_pool=True)#

Thread island.

This class is a user-defined island (UDI) that will run evolutions in a separate thread of execution. Evolution tasks running on this UDI must involve algorithm and problem instances that provide at least the basic thread_safety guarantee, otherwise errors will be raised during the evolution.

Note that algorithms and problems implemented in Python are never considered thread safe, and thus this UDI can be used only with algorithms and problems implemented in C++.

The use_pool flag signals whether or not this island should use a common thread pool shared by all islands.

Using a thread pool is more computationally-efficient, for at least two reasons:

  • it avoids runtime overhead when the number of islands evolving simultaneously is larger than the CPU count (e.g., in a large archipelago);

  • because the implementation uses the Intel TBB libraries, it integrates better with other pagmo facilities built on top of TBB (e.g., the thread_bfe batch fitness evaluator).

A thread pool however also introduces a serializing behaviour because the number of evolutions actually running at the same time is limited by the CPU count (whereas without the thread pool an unlimited number of evolutions can be active at the same time, albeit with a performance penalty).

See also the documentation of the corresponding C++ class pagmo::thread_island.

New in version 2.16.0: The use_pool flag.

Parameters

use_pool (bool) – a boolean flag signalling whether or not a thread pool should be used by the island