The pyoptgra module

class pyoptgra.optgra(*args)

This class is a user defined algorithm (UDA) providing a wrapper around OPTGRA, which is written in Fortran.

It is specifically designed for near-linear optimization problems with many constraints. When optimizing a problem, Optgra will first move towards satisfying the constraints, then move along the feasible region boundary to optimize the merit function, fixing constraint violations as they occur.

For this, constraints and the merit function are linearized. Optgra will perform less well on very non-linear merit functions or constraints.

Example:

>>> import pygmo
>>> import pyoptgra
>>> prob = pygmo.problem(pygmo.schwefel(30))
>>> pop = pygmo.population(prob, 1)
>>> algo = pygmo.algorithm(pyoptgra.optgra())
>>> pop = algo.evolve(pop)

Initialize a wrapper instance for the OPTGRA algorithm.

Some of the construction arguments, for example the scaling factors, depend on the dimension of the problem. Passing a problem with a different dimension to the instance’s evolve function will result in an error.

Some problem-specific options are deduced from the problem in the population given to the evolve function.

Parameters:

max_iterations – maximum number of total iterations
max_correction_iterations – number of constraint correction iterations in the beginning If no feasible solution is found within that many iterations, Optgra aborts
max_distance_per_iteration – maximum scaled distance traveled in each iteration
perturbation_for_snd_order_derivatives – Used as delta for numerically computing second order errors of the constraints in the optimization step
variable_scaling_factors – optional - Scaling factors for the input variables. If passed, must be positive and as many as there are variables
variable_types – optional - Flags to set variables to either free (0) or fixed (1). Fixed variables are also called parameters in sensitivity analysis. If passed, must be as many flags as there are variables
constraint_priorities – optional - lower constraint priorities are fulfilled earlier. During the initial constraint correction phase, only constraints with a priority at most k are considered in iteration k. Defaults to zero, so that all constraints are considered from the beginning.
bounds_to_constraints – optional - if true (default), translate box bounds of the given problems into inequality constraints for optgra. Note that when also passing constraint priorities, the original constraints of the problem come first, followed by those derived from the lower box bounds, then those from the upper box bounds. Infinite bounds are ignored and not counted.
bound_constraints_tolerance – optional - constraint tolerance for the constraints derived from bounds
merit_function_threshold – optional - convergence threshold for merit function
force_bounds – optional - whether to force the bounds given by the problem. If false (default), the fitness function might also be called with values of x that are outside of the bounds. Set to true if the fitness function cannot handle that. If active, the gradients evaluated near the bounds will be inacurate potentially leading to convergence issues.
khan_bounds –
optional - whether to gracefully enforce bounds on the decision vector using Michael Khan’s method, by default False.:

\[x = \frac{x_{max} + x_{min}}{2} + \frac{x_{max} - x_{min}}{2} \cdot \sin(x_{Khan})\]

Where \(x\) is the pagmo decision vector and \(x_{Khan}\) is the decision vector passed to OPTGRA. In this way parameter bounds are guaranteed to be satisfied, but the gradients near the bounds approach zero. Pyoptgra uses a variant of the above method that additionally scales the argument of the \(\sin\) function such that the derivatives \(\frac{d x_{Khan}}{d x}\) are unity in the center of the box bounds. Alternatively, to a \(\sin\) function, also a \(\tanh\) or Fourier expansion of a triangle wave can be used as a Khan function. Valid input values are: True (same as ‘sin’),’sin’, ‘tanh’, ‘triangle1’, ‘triangle2’, ‘triangle3’… and False.
optimization_method – select 0 for steepest descent, 1 for modified spectral conjugate gradient method, 2 for spectral conjugate gradient method and 3 for conjugate gradient method
log_level – Control the original screen output of OPTGRA. 0 has no output, 4 and higher have maximum output`. Set this to 0 if you want to use the pygmo logging system based on set_verbosity().

Raises:

ValueError – if optimization_method is not one of 0, 1, 2, or 3 ValueError: if any of
max_iterations, max_correction_iterations, max_distance_per_iteration, – or perturbation_for_snd_order_derivatives are negative

evolve(self, population pop)

Call OPTGRA with the best-fitness member of the population as start value.

Parameters:

population – The population containing the problem and a set of initial solutions.

Returns:

The changed population.

Raises:

ValueError – If the population is empty
ValueError – If the problem contains multiple objectives
ValueError – If the problem is stochastic
ValueError – If the problem dimensions don’t fit to constraint_priorities or variable_scaling_factors that were passed to the wrapper constructor
ValueError – If the problem has finite box bounds and bounds_to_constraints was set to True in the wrapper constructor (default), constraint_priorities were also passed but don’t cover the additional bound-derived constraints

prepare_sensitivity(self, problem, x)

Prepare OPTGRA for sensitivity analysis at x. This is independant from previous and later calls to evolve, but enables calls to sensitivity_matrices, linear_update_new_callable and linear_update_delta on this instance.

This works by creating a linearization of the problem’s fitness function around x.

Parameters:

problem – The problem being analyzed. x: The value of x around which linearization is
performed

Raises:

ValueError – If the problem contains multiple objectives ValueError: If the problem is
stochastic ValueError – If the problem dimensions don’t fit to constraint_priorities or variable_scaling_factors that were passed to the wrapper constructor
ValueError – If the problem has finite box bounds and bounds_to_constraints was set to True in the wrapper constructor (default), constraint_priorities were also passed but don’t cover the additional bound-derived constraints

sensitivity_matrices(self)

Get stored sensitivity matrices prepared by earlier call to prepare_sensivitity. Note that active constraints are constraints that are currently fulfilled, but could be violated in the next iteration. Parameters refer to variables whose variable type was declared as fixed.

Returns:: a boolean list of whether each constraint is active, the sensitivity of constraints + merit function with respect to active constraints, the sensitivity of constraints + merit function with respect to parameters, the sensitivity of variables with respect to active constraints, and the sensitivity of variables with respect to parameters.
Return type:: A tuple of one list and four matrices
Raises:: RuntimeError – If prepare_sensitivity has not been called on this instance

linear_update_new_callable(self, problem)

Perform a single optimization step on the stored value of x, but with a new callable

Parameters:

problem – A problem containing the new callable. Has to have same dimensions and types as the problem passed to prepare_sensitivity

Returns:

tuple of new_x, new_y

Raises:

RuntimeError – If prepare_sensitivity has not been called on this instance ValueError: If
number or type of constraints of the new problem are different from – those of the problem passed to prepare_sensitivity

linear_update_delta(self, constraint_delta)

Perform a single optimization step on the linear approximation prepared with prepare_sensitivity. For this, no new function calls to the problem callable are performed, making this potentially very fast.

Parameters:

constraint_delta – A list of deltas against the constraints. They are subtracted from the
values. (stored)

Returns:

tuple of new_x, new_y

Raises:

RuntimeError – If prepare_sensitivity has not been called on this instance ValueError: If
number of deltas does not fit number of constraints. –

The optgra C++ wrapper

std::tuple<std::vector<double>, std::vector<double>, int> optgra::optimize(const std::vector<double> &initial_x, const std::vector<int> &constraint_types, fitness_callback fitness, gradient_callback gradient, bool has_gradient, int max_iterations = 150, int max_correction_iterations = 90, double max_distance_per_iteration = 10, double perturbation_for_snd_order_derivatives = 1, std::vector<double> convergence_thresholds = {}, std::vector<double> variable_scaling_factors = {}, std::vector<int> constraint_priorities = {}, std::vector<std::string> variable_names = {}, std::vector<std::string> constraint_names = {}, int optimization_method = 2, int derivatives_computation = 1, std::vector<double> autodiff_deltas = {}, std::vector<int> variable_types = {}, int log_level = 1, int verbosity = 0)

Main C++ wrapper function.

Call optgra to optimize a problem. Most of the parameters are identical to the constructor arguments of pyoptgra, but some additional ones are available.

Parameters:

initial_x – the initial guess for the decision vector
constraint_types – types of constraints. Set 0 for equality constraints, -1 for inequality constraints that should be negative, 1 for positive inequality constraints and -2 for unenforced constraints. Last element describes the merit function, with -1 for minimization problems and 1 for maximization problems.
fitness – a callable for the fitness values. It is called with the current x, expected output is an array of first all equality constraints, then all inequality constraints, and last the merit function
gradient – a callable for the gradient values, optional. It is called with the current x, expected output is a two-dimensional array g, with g_ij being the gradient of constraint i with respect to input variable j.
has_gradient – whether the problem has a gradient. If set to False, the gradient callable will not be called and numerical differentiation will be used instead.
max_iterations – the maximum number of iterations. Optional, defaults to 150.
max_correction_iterations – number of constraint correction iterations in the beginning. If no feasible solution is found within that many iterations, Optgra aborts. Optional, defaults to 90.
max_distance_per_iteration – maximum scaled distance traveled in each iteration. Optional, defaults to 10
perturbation_for_snd_order_derivatives – used as delta for numerically computing second order errors of the constraints in the optimization step. Parameter VARSND in Fortran. Optional, defaults to 1
convergence_thresholds – tolerance a constraint can deviate and still be considered fulfilled. Constraints with lower thresholds will be prioritized during optimization. Thresholds of 0 break the optimization process.
variable_scaling_factors – scaling factors for the input variables. If passed, must be positive and as many as there are variables
constraint_priorities – filter in which to consider constraints. Lower constraint priorities are fulfilled earlier. During the initial constraint correction phase, only constraints with a priority at most k are considered in iteration k. Defaults to zero, so that all constraints are considered from the beginning.
variable_names – Not yet implemented
constraint_names – Not yet implemented
optimization_method – select 0 for steepest descent, 1 for modified spectral conjugate gradient method, 2 for spectral conjugate gradient method and 3 for conjugate gradient method. Parameter OPTMET in Fortran.
derivatives_computation – method to compute gradients. 0 is no gradient, 1 is the user-defined gradient function, 2 is a numerical gradient with double differencing, 3 a numerical gradient with single differencing. Parameter VARDER in Fortran.
autodiff_deltas – deltas used for each variable when computing the gradient numerically. Optional, defaults to 0.001.
variable_types – Optional array, specifiying 0 (normal, default) or 1 (fixed, only used for sensitivity) for each variable.
log_level – original OPTGRA logging output: 0 has no output, 4 and higher have maximum output. Set this to 0 if you want to use the pygmo logging system based on set_verbosity().
verbosity – pygmo-style logging output: 0 has no output, N means an output every Nth cost function evaluation. Set log_level to zero to use this.

Throws:

unspecified – any exception thrown by memory errors in standard containers

Returns:

a tuple of the best value of x, the fitness of that x, and a status flag of optgra

std::tuple<std::vector<int>, std::vector<std::vector<double>>, std::vector<std::vector<double>>, std::vector<std::vector<double>>, std::vector<std::vector<double>>> optgra::compute_sensitivity_matrices(const std::vector<double> &x, const std::vector<int> &constraint_types, fitness_callback fitness, gradient_callback gradient, bool has_gradient, double max_distance_per_iteration = 10, double perturbation_for_snd_order_derivatives = 1, std::vector<double> variable_scaling_factors = {}, std::vector<std::string> variable_names = {}, std::vector<std::string> constraint_names = {}, int derivatives_computation = 1, std::vector<double> autodiff_deltas = {}, std::vector<int> variable_types = {}, int log_level = 1, int verbosity = 0)

Compute sensitivity state and matrices in one go, without creating a sensitivity state tuple.

Parameters:

x – the decision vector around which the sensitivity analysis is to be performed
constraint_types – types of constraints. Set 0 for equality constraints, -1 for inequality constraints that should be negative, 1 for positive inequality constraints and -2 for unenforced constraints Last element describes the merit function, with -1 for minimization problems and 1 for maximization problems.
fitness – a callable for the fitness values. It is called with the current x, expected output is an array of first all equality constraints, then all inequality constraints, and last the merit function
gradient – a callable for the gradient values, optional. It is called with the current x, expected output is a two-dimensional array g, with g_ij being the gradient of constraint i with respect to input variable j.
has_gradient – whether the problem has a gradient. If set to False, the gradient callable will not be called and numerical differentiation will be used instead.
max_distance_per_iteration – maximum scaled distance traveled in each iteration. Optional, defaults to 10
perturbation_for_snd_order_derivatives – used as delta for numerically computing second order errors of the constraints in the optimization step. Parameter VARSND in Fortran. Optional, defaults to 1
variable_scaling_factors – scaling factors for the input variables. If passed, must be positive and as many as there are variables
variable_names – Not yet implemented
constraint_names – Not yet implemented
derivatives_computation – method to compute gradients. 0 is no gradient, 1 is the user-defined gradient function, 2 is a numerical gradient with double differencing, 3 a numerical gradient with single differencing. Parameter VARDER in Fortran.
autodiff_deltas – deltas used for each variable when computing the gradient numerically. Optional, defaults to 0.001.
variable_types – Optional array, specifiying 0 (normal, default) or 1 (fixed, only used for sensitivity) for each variable.
log_level – original OPTGRA logging output: 0 has no output, 4 and higher have maximum output
verbosity – pygmo-style logging output: 0 has no output, N means an output every Nth cost function evaluation

Returns:

A tuple of one list and four matrices: a boolean list of whether each constraint is active, the sensitivity of constraints + merit function with respect to active constraints, the sensitivity of constraints + merit function with respect to parameters, the sensitivity of variables with respect to active constraints, and the sensitivity of variables with respect to parameters.

std::tuple<sensitivity_state, std::vector<double>> optgra::prepare_sensitivity_state(const std::vector<double> &x, const std::vector<int> &constraint_types, fitness_callback fitness, gradient_callback gradient, bool has_gradient, double max_distance_per_iteration = 10, double perturbation_for_snd_order_derivatives = 1, std::vector<double> variable_scaling_factors = {}, int derivatives_computation = 1, std::vector<double> autodiff_deltas = {}, std::vector<int> variable_types = {}, int log_level = 1, int verbosity = 0)

Create a state tuple usable for sensitivity updates.

Parameters:

x – the decision vector around which the sensitivity analysis is to be performed
constraint_types – types of constraints. Set 0 for equality constraints, -1 for inequality constraints that should be negative, 1 for positive inequality constraints and -2 for unenforced constraints Last element describes the merit function, with -1 for minimization problems and 1 for maximization problems.
fitness – a callable for the fitness values. It is called with the current x, expected output is an array of first all equality constraints, then all inequality constraints, and last the merit function
gradient – a callable for the gradient values, optional. It is called with the current x, expected output is a two-dimensional array g, with g_ij being the gradient of constraint i with respect to input variable j.
has_gradient – whether the problem has a gradient. If set to False, the gradient callable will not be called and numerical differentiation will be used instead.
max_distance_per_iteration – maximum scaled distance traveled in each iteration. Optional, defaults to 10
perturbation_for_snd_order_derivatives – used as delta for numerically computing second order errors of the constraints in the optimization step. Parameter VARSND in Fortran. Optional, defaults to 1
variable_scaling_factors – scaling factors for the input variables. If passed, must be positive and as many as there are variables
derivatives_computation – method to compute gradients. 0 is no gradient, 1 is the user-defined gradient function, 2 is a numerical gradient with double differencing, 3 a numerical gradient with single differencing. Parameter VARDER in Fortran.
autodiff_deltas – deltas used for each variable when computing the gradient numerically. Optional, defaults to 0.001.
variable_types – Optional array, specifiying 0 (normal, default) or 1 (fixed, only used for sensitivity) for each variable.
log_level – original OPTGRA logging output: 0 has no output, 4 and higher have maximum output
verbosity – pygmo-style logging output: 0 has no output, N means an output every Nth cost function evaluation

Returns:

A tuple of the current sensitivity state and the x for which the sensitivity analysis was performed. It may be different from the x given as argument, if optimization steps were performed in the meantime.

std::tuple<std::vector<int>, std::vector<std::vector<double>>, std::vector<std::vector<double>>, std::vector<std::vector<double>>, std::vector<std::vector<double>>> optgra::get_sensitivity_matrices(sensitivity_state state_tuple, const std::vector<int> &variable_types, vector<int> constraint_types, double max_distance_per_iteration = 10): Compute sensitivity matrics from a sensitivity state tuple.

std::tuple<std::vector<double>, std::vector<double>, int> optgra::sensitivity_update_new_callable(sensitivity_state state_tuple, const std::vector<int> &variable_types, const std::vector<int> &constraint_types, fitness_callback fitness, gradient_callback gradient, bool has_gradient, double max_distance_per_iteration = 10, double perturbation_for_snd_order_derivatives = 1, std::vector<double> variable_scaling_factors = {}, int derivatives_computation = 1, std::vector<double> autodiff_deltas = {}, int log_level = 1, int verbosity = 0): Perform one optimization step with a new fitness callable, starting from the value of x that was set with prepare_sensitivity_state.

std::tuple<std::vector<double>, std::vector<double>, int> optgra::sensitivity_update_constraint_delta(sensitivity_state state_tuple, const std::vector<int> &variable_types, const std::vector<int> &constraint_types, vector<double> &delta, double max_distance_per_iteration = 10, double perturbation_for_snd_order_derivatives = 1, std::vector<double> variable_scaling_factors = {}, int log_level = 1, int verbosity = 0): Perform an update step based on the prepared sensitivity state, without any calls to the fitness callbacks.