Module Documentation

File: __main__.py

boss.__main__.main(args=None)

The main routine.

File: bo/acq/base.py

Base class for acquisition functions used in Bayesian optimization.

To add a new acquisition function:

1. Create a class that inherits from BaseAcquisition and implement all methods, that are decorated with @abstractmethod.

2. Modify ‘select_acqfn’ in the boss/bo/acq/factory.py module accordingly.

class boss.bo.acq.base.BaseAcquisition(model=None)

Base class for acquisition functions used in Bayesian optimization.

This class is intended to be subclassed. It contains functionalities for evaluating and minimizing an acquisition function.

Attributes:

has_gradient

Whether the acquisition function has gradient.

model

Property, acquisition functions require access to the model.

Methods

__call__(x)	Call self as a function.
evaluate(x)	Abstract method.
evaluate_with_gradient(x)	Abstract method.
minimize(bounds[, optimtype])	Minimizes the acquisition function to find the next sampling location 'x_next'.

abstract evaluate(x)

Abstract method. Evaluates the acquisition function at x and returns its value.

Parameters:

xndarray

Location to evaluate acquisition function at

Returns:

f_acqndarray

Acquisition function evaluated at ‘x’. 2D array containing data with ‘float’ type.

abstract evaluate_with_gradient(x)

Abstract method. Evaluates the acquisition function at x and returns its value and gradient.

Parameters:

xndarray

Location to evaluate acquisition functions at.

Returns:

tuple of ndarray

Acquisition function evaluation and it’s gradient at ‘x’. 2D arrays with ‘float’ type.

abstract property has_gradient

Whether the acquisition function has gradient.

minimize(bounds, optimtype='score')

Minimizes the acquisition function to find the next sampling location ‘x_next’.

Parameters:

dimint

Dimension of space of the user function.

boundsndarray

Bounds of used variables, defining the space of the user function. 2D array containing data with ‘float’ type.

Returns:

ndarray

Array containing found minimum with ‘float’ type.

property model

Property, acquisition functions require access to the model.

Returns:

Model

BaseModel used for the optimization.

class boss.bo.acq.base.CostAwareAcquisition(model=None)

Base class for cost-aware acquisition functions.

Attributes:

has_gradient

Whether the acquisition function has gradient.

model

Property, acquisition functions require access to the model.

Methods

__call__(x)	Call self as a function.
evaluate(x)	Abstract method.
evaluate_cost(x)	Evaluates the acquisition cost at x and returns its value.
evaluate_with_gradient(x)	Abstract method.
minimize(bounds[, optimtype])	Minimizes the acquisition function to find the next sampling location 'x_next'.

abstract evaluate_cost(x)

Evaluates the acquisition cost at x and returns its value.

Parameters:

xndarray

Location to evaluate acquisition cost at

Returns:

costfloat

File: bo/acq/ei.py

class boss.bo.acq.ei.EI(model=None, jitter=0.0)

Expected improvement acquisition function.

Takes one parameter. Parameter values > 0 boost exploration.

Attributes:

has_gradient

Property to inform whether acquisition function has a gradient.

model

Property, acquisition functions require access to the model.

Methods

__call__(x)	Call self as a function.
evaluate(x)	Compute the expected improvement at location x.
evaluate_with_gradient(x)	Compute the expected improvement and its gradient at location x.
minimize(bounds[, optimtype])	Minimizes the acquisition function to find the next sampling location 'x_next'.

evaluate(x)

Compute the expected improvement at location x.

Parameters:

xndarray

Input to the acquisition function. 2D array containing data with ‘float’ type.

Returns:

f_acqndarray

Acquisition function evaluated at ‘x’. 2D array containing data with ‘float’ type.

evaluate_with_gradient(x)

Compute the expected improvement and its gradient at location x. Gradients are required to apply L-BFGS-B minimizer.

Parameters:

xndarray

Input to the acquisition function. 2D array containing data with ‘float’ type.

Returns:

f_acq, df_acqtuple of ndarray

Acquisition function and it’s gradient, evaluated at ‘x’. Tuple of 2D arrays, containing data with ‘float’ type.

property has_gradient

Property to inform whether acquisition function has a gradient.

Returns:

bool

File: bo/acq/elcb.py

class boss.bo.acq.elcb.ELCB(model=None)

GP-Lower Confidence Bound acquisition function with increasing exploration.

Doesn’t take any parameters. The exploration weight is given by

weight = sqrt( 2*log[ ( i^((dim/2) + 2)*pi^(2) ) / ( 3*0.1 ) ] )

The implementation is based on the following papers

(1) N. Srinivas, A. Krause, S. M. Kakade, and M. Seeger. Gaussian process optimization in the bandit setting: No regret and experimental design. Proc. ICML, 2010

(2) E. Brochu, V. M. Cora, and N. de Freitas. A tutorial on Bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning. arXiv:1012.2599, 2010,

where the delta parameter introduced in Brochu et al. has been set to 0.1.

Attributes:

has_gradient

Property to inform whether acquisition function has a gradient.

model

Property, acquisition functions require access to the model.

Methods

__call__(x)	Call self as a function.
evaluate(x)	Compute the ELCB acqfn at location x.
evaluate_with_gradient(x)	Compute the ELCB acqfn and its gradient at location x.
minimize(bounds[, optimtype])	Minimizes the acquisition function to find the next sampling location 'x_next'.

_get_explore_weight()

Calculate exploration weight for ELCB acquisition function.

The exploration weight depends on the amount of samples and the dimension. The derivation is described in the Article, cited in the class docstring.

Returns:

int

Exploration weight.

evaluate(x)

Compute the ELCB acqfn at location x.

Parameters:

xndarray

Input to the acquisition function. 2D array containing data with ‘float’ type.

Returns:

f_acqndarray

Acquisition function evaluated at ‘x’. 2D array containing data with ‘float’ type.

evaluate_with_gradient(x)

Compute the ELCB acqfn and its gradient at location x. Gradients are required to apply L-BFGS-B minimizer.

Parameters:

xndarray

Input to the acquisition function. 2D array containing data with ‘float’ type.

Returns:

f_acq, df_acqtuple of ndarray

Acquisition function and it’s gradient, evaluated at ‘x’. Tuple of 2D arrays, containing data with ‘float’ type.

property has_gradient

Property to inform whether acquisition function has a gradient.

Returns:

bool

File: bo/acq/exploit.py

class boss.bo.acq.exploit.Exploit(model=None)

Purely exploiting acquisition function

Doesn’t take any parameters.

Attributes:

has_gradient

Property to inform whether acquisition function has a gradient.

model

Property, acquisition functions require access to the model.

Methods

__call__(x)	Call self as a function.
evaluate(x)	Compute the exploitation acqfn at location x.
evaluate_with_gradient(x)	Compute the exploitation acqfn and its gradient at location x.
minimize(bounds[, optimtype])	Minimizes the acquisition function to find the next sampling location 'x_next'.

evaluate(x)

Compute the exploitation acqfn at location x.

Parameters:

xndarray

Input to the acquisition function. 2D array containing data with ‘float’ type.

Returns:

f_acqndarray

Acquisition function evaluated at ‘x’. 2D array containing data with ‘float’ type.

evaluate_with_gradient(x)

Compute the exploitation acqfn and its gradient at location x. Gradients are required to apply L-BFGS-B minimizer.

Parameters:

xndarray

Input to the acquisition function. 2D array containing data with ‘float’ type.

Returns:

f_acq, df_acqtuple of ndarray

Acquisition function and it’s gradient, evaluated at ‘x’. Tuple of 2D arrays, containing data with ‘float’ type.

property has_gradient

Property to inform whether acquisition function has a gradient.

Returns:

bool

File: bo/acq/explore.py

class boss.bo.acq.explore.Explore(model=None)

Purely exploring acquisition function.

Doesn’t take any parameters.

Attributes:

has_gradient

Property to inform whether acquisition function has a gradient.

model

Property, acquisition functions require access to the model.

Methods

__call__(x)	Call self as a function.
evaluate(x)	Compute the exploration acqfn at location x.
evaluate_with_gradient(x)	Compute the exploration acqfn and its gradient at location x.
minimize(bounds[, optimtype])	Minimizes the acquisition function to find the next sampling location 'x_next'.

evaluate(x)

Compute the exploration acqfn at location x.

Parameters:

xndarray

Input to the acquisition function. 2D array containing data with ‘float’ type.

Returns:

f_acqndarray

Acquisition function evaluated at ‘x’. 2D array containing data with ‘float’ type.

evaluate_with_gradient(x)

Compute the exploration acqfn and its gradient at location x. Gradients are required to apply L-BFGS-B minimizer.

Parameters:

xndarray

Input to the acquisition function. 2D array containing data with ‘float’ type.

Returns:

f_acq, df_acqtuple of ndarray

Acquisition function and it’s gradient, evaluated at ‘x’. Tuple of 2D arrays, containing data with ‘float’ type.

property has_gradient

Property to inform whether acquisition function has a gradient.

Returns:

bool

File: bo/acq/lcb.py

class boss.bo.acq.lcb.LCB(model=None, weight=2.0)

Lower Confidence Bound (LCB) acquisition function with constant exploration weight.

Takes exploration weight as parameter.

Attributes:

has_gradient

Property to inform whether acquisition function has a gradient.

model

Property, acquisition functions require access to the model.

Methods

__call__(x)	Call self as a function.
evaluate(x)	Compute the LCB acquisition function at x.
evaluate_with_gradient(x)	Compute the LCB acquisition function and its gradient at x.
minimize(bounds[, optimtype])	Minimizes the acquisition function to find the next sampling location 'x_next'.

evaluate(x)

Compute the LCB acquisition function at x.

Parameters:

xndarray

Input to the acquisition function. 2D array containing data with ‘float’ type.

Returns:

f_acqndarray

Acquisition function evaluated at ‘x’. 2D array containing data with ‘float’ type.

evaluate_with_gradient(x)

Compute the LCB acquisition function and its gradient at x. Gradients are required to apply L-BFGS-B minimizer.

Parameters:

xndarray

Input to the acquisition function. 2D array containing data with ‘float’ type.

Returns:

f_acq, df_acqtuple of ndarray

Acquisition function and it’s gradient, evaluated at ‘x’. Tuple of 2D arrays, containing data with ‘float’ type.

property has_gradient

Property to inform whether acquisition function has a gradient.

Returns:

bool

File: bo/acq/multi.py

class boss.bo.acq.multi.MTAcquisition(cost_arr, target_index=0, model=None)

Base class for multi-task acquisition functions.

Attributes:

model

Property, acquisition functions require access to the model.

Methods

__call__(x)	Call self as a function.
evaluate(x)	Abstract method.
evaluate_cost(x)	Evaluates the acquisition cost at x and returns its value.
evaluate_with_gradient(x)	Evaluates the acquisition function at x and returns its value and gradient
has_gradient()	Property to inform whether acquisition function has a gradient.
minimize(bounds[, optimtype])	Minimizes the acquisition function to find the next sampling location 'x_next'.

evaluate_cost(x)

Evaluates the acquisition cost at x and returns its value.

Parameters:

xndarray

Location to evaluate acquisition cost at

Returns:

costfloat

evaluate_with_gradient(x)

Evaluates the acquisition function at x and returns its value and gradient

Parameters:

xndarray

Location to evaluate acquisition functions at.

Returns:

tuple of ndarray

has_gradient()

Property to inform whether acquisition function has a gradient.

Returns:

bool

class boss.bo.acq.multi.MTHeuristic(acqfn, cost_arr, model=None)

Heuristic acquisition rules for multi-task optimisation.

Attributes:

model

Property, acquisition functions require access to the model.

Methods

__call__(x)	Call self as a function.
evaluate(x)	Evaluates the acquisition function at x and returns its value.
evaluate_cost(x)	Evaluates the acquisition cost at x and returns its value.
evaluate_with_gradient(x)	Evaluates the acquisition function at x and returns its value and gradient
has_gradient()	Property to inform whether acquisition function has a gradient.
minimize(bounds[, optimtype])	Minimizes the acquisition function to find the next sampling location 'x_next'.

evaluate(x)

Evaluates the acquisition function at x and returns its value.

Parameters:

xndarray

Location to evaluate acquisition function at

Returns:

f_acqndarray

minimize(bounds, optimtype='score')

Minimizes the acquisition function to find the next sampling location ‘x_next’.

Parameters:

boundsndarray

Bounds of used variables, defining the space of the user function. 2D array containing data with ‘float’ type.

Returns:

ndarray

Array containing found minimum with ‘float’ type.

property model

Property, acquisition functions require access to the model.

Returns:

Model

BaseModel used for the optimization.

File: bo/bo_main.py

class boss.bo.bo_main.BOMain(f, bounds, **keywords)

Class for handling Bayesian Optimization

Methods

acquire()	Gets the next acquisition.
exceeds_cost_limit(X_next)	Checks whether the next acquisition would exceed cost limit.
from_file(ipfile[, outfile, f])	Initialize BOMain from a BOSS input or rst file.
from_settings(settings[, rst_data])	Construction from a Settings object.
get_initpts()	If initial data is not provided, get it from the rst/init manager.
init_model(X, Y[, params])	Initializes the GP model.
init_run(X_init[, Y_init])	Evalutes initial points and calls model initialization.
run([X_init, Y_init, iterpts, maxcost])	The Bayesian optimization main loop.

_eval_user_func(X)

Evalutes the userfn and writes data to the rstfile.

_setup()

Common setup for all factory methods.

_update_model(X_next, Y_next)

Optimizes and refits model with new data.

_update_results(X_next)

Updates BOResults with acqs, model and min info.

acquire()

Gets the next acquisition.

exceeds_cost_limit(X_next)

Checks whether the next acquisition would exceed cost limit.

classmethod from_file(ipfile, outfile=None, f=None, **new_keywords)

Initialize BOMain from a BOSS input or rst file.

Parameters:

ipfilepath_like

The input file to initialize from, can be either a boss input or rst file.

**new_keywords

Any new BOSS keywords.

classmethod from_settings(settings, rst_data=None)

Construction from a Settings object.

get_initpts()

If initial data is not provided, get it from the rst/init manager.

init_model(X, Y, params=None)

Initializes the GP model.

init_run(X_init, Y_init=None)

Evalutes initial points and calls model initialization.

This method assumes that, at least, initial X-values have been determined, i.e. passed by the user or retrieved from file or an InitManager (see resolve_initpts).

Parameters:

X_init: np.ndarray

Initial X-values, must be completely specified, i.e., cannot be None nor have nan-elements.

Y_init: Optional[np.ndarray]

Initial Y-values, can be omitted entirely or partially, in which case the user function will be evaluated at the missing X-locations.

Returns:

X_next: np.ndarray

The very first acquisition that will be used in the BO-run.

run(X_init=None, Y_init=None, iterpts=None, maxcost=None)

The Bayesian optimization main loop. Evaluates first the initialization points, then creates a GP model and uses it and an acquisition function to locate the next points where to evaluate. Stops when a pre-specified number of initialization points and BO points have been acquired or a convergence criterion or cost limit is met.

Parameters:

X_initOptional[ArrayLike]

Initial input points, provided as rows in a 2D array, to use for the BO. Will take precedence over any other specification of initial points coming from initmanagers or restart files.

Y_initOptional[ArrayLike]

Precomputed output data corresponding to X_init. Can contain fewer data points than X_init, in which case the userfn will be called instead.

iterptsOptional[int]

The maximum number of BO iterations that will be performed unless another termination criterion is fulfilled first. This option, if set, will take precedence over the iterpts specified in settings.

maxcostOptional[float]

A cost limit for user function evaluations that, if provided, will be used to terminate the BO-loop.

Returns:

BOResults

Provides access to the most important results from the optimization.

File: bo/initmanager.py

class boss.bo.initmanager.InitManager(inittype, bounds, initpts, seed=None, scramble=False)

Methods

get_all()	Returns all generated initial points
get_x(i)	Returns the i:th initial point

_extend(data, npts)

Extend initial points with task index.

_grid(bounds, npts)

Initial points in a grid. Total number of points returned is npts^dim.

_make_grid(bounds, npts)

Return grid with npts points across bounds in each dimension.

_random(bounds, initpts)

Initial points randomly

_sobol(bounds, initpts, scramble=False)

Initial points with the quasi-random Sobol sequence

get_all()

Returns all generated initial points

get_x(i)

Returns the i:th initial point

File: bo/kernel_factory.py

boss.bo.kernel_factory._add_coreg(kernel: Kern, settings: Settings) → Kern

Extends a basic kernel with coregionalisation.

boss.bo.kernel_factory._apply_coreg_settings(kernel_multi: Kern, settings: Settings) → None

Sets coregionalisation values and priors.

boss.bo.kernel_factory._resolve_thepriorpar(settings: Settings) → ndarray

Resolve the prior distribution values of the hyperparameters based on the settings.

Parameters: settings: Settings object containing configuration for initialization.

Returns: list: A nested list of prior distribution values for each hyperparameter.

boss.bo.kernel_factory._resolve_thetainit(settings: Settings) → ndarray

Resolve the initial values of theta parameters based on the settings.

Parameters: settings: Settings object containing configuration for initialization.

Returns: list: A list of initial values for theta parameters.

boss.bo.kernel_factory._set_gamma_prior(kernel: Kern, settings: Settings) → None

Sets a gamma prior on the kernel.

boss.bo.kernel_factory._set_hyper_constraints(kernel: Kern, settings: Settings) → None

Sets hyperparameter constraints on kernels.

boss.bo.kernel_factory._set_hyper_priors(kernel: Kern, settings: Settings) → None

Sets hyperparameter priors on kernels.

boss.bo.kernel_factory._set_hyper_values(kernel: Kern, settings: Settings) → None

Sets hyperparameter values

boss.bo.kernel_factory.init_kernel(settings: Settings) → Kern

Creates a kernel according to the settings.

The kernel will be a product kernel with parts specified by the kernel keyword, no priors or constraints will be set.

boss.bo.kernel_factory.select_kernel(settings: Settings) → Kern

Convenience function to initialize a kernel and apply hyperparameters.

File: bo/model.py

class boss.bo.model.MTModel(kernel: Kern, X: _SupportsArray[dtype] | _NestedSequence[_SupportsArray[dtype]] | bool | int | float | complex | str | bytes | _NestedSequence[bool | int | float | complex | str | bytes], Y: _SupportsArray[dtype] | _NestedSequence[_SupportsArray[dtype]] | bool | int | float | complex | str | bytes | _NestedSequence[bool | int | float | complex | str | bytes], noise: float = 1e-12, ynorm: bool = False)

Functionality for creating, refitting and optimizing a multi-task GP model.

Attributes:

X

Y

dim

inds

kernel

Methods

add_data(X_new, Y_new)	Updates the model evidence (observations) dataset appending.
check_task_indices(inds)	Raises an error if all tasks are not included in the index list or if the list includes more tasks than expected.
estimate_num_local_minima(search_bounds)	Returns estimated number of local minima calculated based on model properties.
extend_input(x, index)	Returns x extended with task index.
get_X([index])	Returns observed X.
get_Y([index])	Returns observed Y.
get_all_params()	Returns model parameters as a dictionary with entries: noise, lengthscales, periods, kappa, W There exists a period only for those dimensions which are using a periodic kernel.
get_best_xy([index])	Returns the lowest energy acquisitions (x, y).
get_task_covariance()	Returns estimated task covariance matrix.
get_unfixed_params()	Returns the unfixed parameters of the model in an array.
optimize([restarts])	Updates the model hyperparameters by maximizing marginal likelihood.
predict(X[, index, noise, norm])	Returns model prediction mean and variance at point x, with or without model variance (noise) and normalisation (norm).
predict_grads(X[, index, norm])	Returns model prediction mean and variance gradients with respect to input at point x, with or without normalisation (norm).
predict_mean_grad(X[, index, norm])	Returns model mean and its gradient at point x, with or without normalisation (norm).
predict_mean_sd_grads(X[, index, noise, norm])	Returns the model prediction mean, standard deviation and their gradients at point x, with or without model variance (noise) and normalisation (norm).
predict_task_covariance(x)	Return predictive covariance between tasks at point x.
redefine_data(X, Y)	Updates the model evidence (observations) dataset overwriting.
sample_unfixed_params(num_samples)	Sample unfixed model parameters.
set_unfixed_params(params)	Sets the unfixed parameters of the model to given values.

add_data(X_new: _SupportsArray[dtype] | _NestedSequence[_SupportsArray[dtype]] | bool | int | float | complex | str | bytes | _NestedSequence[bool | int | float | complex | str | bytes], Y_new: _SupportsArray[dtype] | _NestedSequence[_SupportsArray[dtype]] | bool | int | float | complex | str | bytes | _NestedSequence[bool | int | float | complex | str | bytes]) → None

Updates the model evidence (observations) dataset appending.

check_task_indices(inds: ndarray[Any, dtype[ScalarType]]) → None

Raises an error if all tasks are not included in the index list or if the list includes more tasks than expected.

estimate_num_local_minima(search_bounds: Any) → int

Returns estimated number of local minima calculated based on model properties.

extend_input(x: _SupportsArray[dtype] | _NestedSequence[_SupportsArray[dtype]] | bool | int | float | complex | str | bytes | _NestedSequence[bool | int | float | complex | str | bytes], index: Any) → ndarray[Any, dtype[ScalarType]]

Returns x extended with task index.

get_X(index: int | None = None) → ndarray[Any, dtype[ScalarType]]

Returns observed X.

get_Y(index: int | None = None) → ndarray[Any, dtype[ScalarType]]

Returns observed Y.

get_all_params() → dict[str, float | list]

Returns model parameters as a dictionary with entries: noise, lengthscales, periods, kappa, W There exists a period only for those dimensions which are using a periodic kernel.

get_best_xy(index: int | None = None) → tuple[numpy.ndarray[typing.Any, numpy.dtype[+ScalarType]], float]

Returns the lowest energy acquisitions (x, y).

get_task_covariance() → ndarray[Any, dtype[ScalarType]]

Returns estimated task covariance matrix.

predict(X: _SupportsArray[dtype] | _NestedSequence[_SupportsArray[dtype]] | bool | int | float | complex | str | bytes | _NestedSequence[bool | int | float | complex | str | bytes], index: _SupportsArray[dtype] | _NestedSequence[_SupportsArray[dtype]] | bool | int | float | complex | str | bytes | _NestedSequence[bool | int | float | complex | str | bytes] | None = None, noise: bool = True, norm: bool = False)

Returns model prediction mean and variance at point x, with or without model variance (noise) and normalisation (norm).

Task index can be included in the input X or provided with index.

predict_grads(X: _SupportsArray[dtype] | _NestedSequence[_SupportsArray[dtype]] | bool | int | float | complex | str | bytes | _NestedSequence[bool | int | float | complex | str | bytes], index: Any | None = None, norm: bool = False) → tuple[numpy.ndarray[typing.Any, numpy.dtype[+ScalarType]], numpy.ndarray[typing.Any, numpy.dtype[+ScalarType]]]

Returns model prediction mean and variance gradients with respect to input at point x, with or without normalisation (norm).

Task index can be included in the input x or provided with index.

predict_mean_grad(X: _SupportsArray[dtype] | _NestedSequence[_SupportsArray[dtype]] | bool | int | float | complex | str | bytes | _NestedSequence[bool | int | float | complex | str | bytes], index: Any | None = None, norm: bool = True) → tuple[numpy.ndarray[typing.Any, numpy.dtype[+ScalarType]], numpy.ndarray[typing.Any, numpy.dtype[+ScalarType]]]

Returns model mean and its gradient at point x, with or without normalisation (norm).

Task index can be included in the input x or provided with index.

predict_mean_sd_grads(X: _SupportsArray[dtype] | _NestedSequence[_SupportsArray[dtype]] | bool | int | float | complex | str | bytes | _NestedSequence[bool | int | float | complex | str | bytes], index: Any | None = None, noise: bool = True, norm: bool = True) → tuple[numpy.ndarray[typing.Any, numpy.dtype[+ScalarType]], numpy.ndarray[typing.Any, numpy.dtype[+ScalarType]], numpy.ndarray[typing.Any, numpy.dtype[+ScalarType]], numpy.ndarray[typing.Any, numpy.dtype[+ScalarType]]]

Returns the model prediction mean, standard deviation and their gradients at point x, with or without model variance (noise) and normalisation (norm).

Task index can be included in the input x or provided with index.

predict_task_covariance(x: _SupportsArray[dtype] | _NestedSequence[_SupportsArray[dtype]] | bool | int | float | complex | str | bytes | _NestedSequence[bool | int | float | complex | str | bytes]) → ndarray[Any, dtype[ScalarType]]

Return predictive covariance between tasks at point x.

redefine_data(X: _SupportsArray[dtype] | _NestedSequence[_SupportsArray[dtype]] | bool | int | float | complex | str | bytes | _NestedSequence[bool | int | float | complex | str | bytes], Y: _SupportsArray[dtype] | _NestedSequence[_SupportsArray[dtype]] | bool | int | float | complex | str | bytes | _NestedSequence[bool | int | float | complex | str | bytes]) → None

Updates the model evidence (observations) dataset overwriting.

File: bo/rstmanager.py

class boss.bo.rstmanager.RstManager(settings, rst_data=None)

A class that handles restart-files (rst-files). These files can be used to

1. introduce acquisition data from another source as initial values,

2. continue a run that has been interrupted for some reason,

3. continue a finished run by acquiring more points or

4. keep the acquisitions but change the model or settings for a rerun.

Attributes:

X

Y

Methods

get_theta(i, n)	Returns the model paramters at iteration i from the rst-data or None if they can't be found.
get_x(i)	Returns the i:th acquisition location from the rst-data or None if it can't be found.
get_y(i)	Returns the i:th acquisition evaluation (energy and gradient) from the rst-data or None if it can't be found.
new_data(x, y)	Outputs a new data point (x,y) to rst file.
new_model_params(mod_param)	Outputs a new set of model parameters to rst file.

new_file

get_theta(i, n)

Returns the model paramters at iteration i from the rst-data or None if they can’t be found.

get_x(i)

Returns the i:th acquisition location from the rst-data or None if it can’t be found.

get_y(i)

Returns the i:th acquisition evaluation (energy and gradient) from the rst-data or None if it can’t be found.

new_data(x, y)

Outputs a new data point (x,y) to rst file.

new_model_params(mod_param)

Outputs a new set of model parameters to rst file.

File: bo/userfunc.py

File: io/dump.py

Functionality to output raw data on request to files outside of the main output (.out) file and the restart (.rst) file.

boss.io.dump.build_query_points(settings, defaults)

Build array of query points.

boss.io.dump.dump_acqfn(settings, dest_file, acqfn, defs)

Outputs acquisition function slice (up to 2D) in a grid to acqfns/it#.dat

boss.io.dump.dump_mep(path)

Outputs the coordinates of each minimum energy path into files

boss.io.dump.dump_model(settings, dest_file, model, mod_params, x_glmin)

Outputs model slice (up to 2D) mean and variance in a grid to models/it#.dat

boss.io.dump.dump_truef(settings, dest_file, last_xhat)

Outputs true function slice (up to 2D) in a grid

File: io/ioutils.py

boss.io.ioutils.append_write(file_path, text)

Writes the text into the given file appending.

boss.io.ioutils.data_line(X, Q=None, fstr='%15.7E', separ='     ')

Returns a line of data. X and Q (optional) should be 1D arrays/lists.

boss.io.ioutils.overwrite(file_path, text)

Writes the text into the given file overwriting.

boss.io.ioutils.read_cols(file_path, skiprows=1)

Reads data columns from a file into a numpy array, where each column is separated like col1 = arr[:,0], col2 = arr[:,1] etc.

boss.io.ioutils.write_cols(file_path, vectorList, space='    ', titleLine=None, formatString='%15.7E')

Needs as input a vector of form [ [column1 elements], [column2 elements], … ]

File: io/main_output.py

class boss.io.main_output.MainOutput(settings, timer=None)

Functionality to write to the main output (*.out) file.

Methods

add_settings(settings)	Outputs the interpreted code variable settings to main output file.
convergence_stop()	Announces BO stop due to global minimum convergence
footer(totaltime)	Writes a footer to main output file
header()	Writes a header to main output file overwriting a possibly existing old output file at the same filepath.
ipfile_repeat(is_rst)	Repeats the input file near the beggining of the main output file.
iteration_summary(results, timer)	Outputs info about one BO iteration to main output file
maxcost_stop()	Announces BO stop due to cost limit reached
mep_result(mep)	Writes the results of MEP
mep_start(mep)	Writes MEP options and local minima
new_file()	Intializes a new main output file with header, input file and settings.
progress_msg(msg[, preceding_bl, nospace])	Announce progress message to main output file depending on verbosity.

initpt_summary
section_header
summarize_results

add_settings(settings)

Outputs the interpreted code variable settings to main output file.

convergence_stop()

Announces BO stop due to global minimum convergence

footer(totaltime)

Writes a footer to main output file

header()

Writes a header to main output file overwriting a possibly existing old output file at the same filepath.

ipfile_repeat(is_rst)

Repeats the input file near the beggining of the main output file.

iteration_summary(results, timer)

Outputs info about one BO iteration to main output file

maxcost_stop()

Announces BO stop due to cost limit reached

mep_result(mep)

Writes the results of MEP

mep_start(mep)

Writes MEP options and local minima

new_file()

Intializes a new main output file with header, input file and settings.

progress_msg(msg, preceding_bl=False, nospace=False)

Announce progress message to main output file depending on verbosity.

File: io/parse.py

The format of the input file is a set of space-separated keyword value pairs:

# comment key1 val1 key2 val2 # comment2

Keyword values are evaluated according to the ruleset:

foo -> str 3 -> int 1.0 -> float foo bar -> List[str] 3 10 -> np.ndarray(shape(1, 2), dtype=int) 5.0 7.0 -> np.ndarray(shape(1, 2), dtype=float) 1.0 2.0 ; 3.0 4.0 -> np.ndarray(shape=(2,2), dtype=float)

boss.io.parse.parse_best_acqs(settings, outfile)

Extracts xbest and ybest from all bo iterations in the output file. Returns a numpy array where each element is [npts, xbest, ybest]. Works only on output files with verbosity levels of at least 1.

boss.io.parse.parse_conv_measures(settings, outfile)

Extracts dx_glmin and dmu_glmin from all bo iterations in the output file. Returns a numpy array where each element is [npts, dx_glmin, dmu_glmin]. Works only on output files with verbosity levels of at least 1.

boss.io.parse.parse_input_file(file_path, skip=None)

Parses a boss input file.

Parses the contents of the file and organizes keywords and their values into a dictionary. If present, restart data present under the optional results header is also parsed.

Parameters:

input_fileOptional[Union[str, Path]]

Input file path.

skip: Optional[str] = None

Skip parsing part of the input, can be either ‘keywords’ or ‘results’.

Returns:

input_data: dict

A dictionary containing the parsed input data. Can contain the following items:

is_rstbool

Whether rst data was present in the input_file or not

rst_data: np.ndarray

Array containing rst data, each row is composed of the X, y data followed by the model parameters. Rows containing the initial points and/or missing y-data are padded with np.nan.

boss.io.parse.parse_min_preds(settings, outfile)

Extracts x_glmin, mu_glmin and nu_glmin from all bo iterations in the output file. Returns a numpy array where each row is [npts, x_glmin, mu_glmin, nu_glmin]. Works on output files with all verbosity levels.

boss.io.parse.parse_minima(outfile)

Extract local minima

boss.io.parse.parse_mod_params(outfile)

Extracts unfixed GP model hyperparameters from all bo iterations in the output file. Returns a numpy array where each element is [npts, unfixed model parameters]. Works only on output files with verbosity levels of at least 2.

boss.io.parse.parse_xnexts(outfile)

Extracts xnext locations from all bo iterations in the output file. Returns a numpy array where each element is [npts, xnext]. Note that this means that the location ‘xnext’ is to be evaluated in the next iteration ‘iter+1’. Works only on output files with verbosity levels of at least 2.

File: keywords.py

This module, together with Settings, implements a global keyword system for BOSS. Here, the types and default values for all keywords are encoded and functions for keyword I/O are implemented (these are used in the BOSS CLI).

Valid BOSS keywords are stored in a global keywords dict named categories. The keywords are further divided into subcategories depending on their dimensionality

and the element type. More specifically, each key in categories is a tuple (type, ndim)

where type is the element type and ndim is the dimensionality (as returned by np.ndim) and the correspondig value is a dict that maps keywords that have this structure to their default values.

Examples

Single booleans have type bool and ndim=0 so all such keywords are grouped under categories[(bool, 0)]. 2D-arrays of floats have type float and ndim=2 so they are grouped under categories[(float, 2)].

To add a new keyword to BOSS you simple need to make a new entry in the correct subcategory below, the keywords will then be automatically available for use in the code via the Settings object.

boss.keywords._eval_bool(x)

Converts string input to booleans.

Boolean values can be specified in BOSS input files as 0 / 1, [y]es / [n]o, [t]rue / [f]alse where all the words are case-insensitive. This function handles conversion from these strings to proper Python booleans.

boss.keywords.destringify(val_str, category)

Converts a string to an appropriate Python object.

When a boss input file is parsed, each string containing the value of a keyword is passed to this function. Strings are evaluated according to the BOSS input ruleset (see BOSS documentation).

Parameters:

val_strstr

An input string to be evaluated.

categoryTuple[type, int]

The target type and dimensionality of the string evaluation. For instance, a string ‘True’ with category (bool, 1) will be evaluated to [True].

Returns:

Any

The result of the string evaluation.

boss.keywords.find_category(key)

Finds the category dict that contains a given key.

boss.keywords.func_to_keyword(func)

The user function is either an ordinary function or a user defined, callable object. In the second case we must pass the type and not the object instance to inspect.getfile.

boss.keywords.get_copied_categories()

Returns deep copies of all categories.

Useful when setting default values using the category dicts and we want to avoid changing the default value contained in the category dict itself.

boss.keywords.stringify(val)

Convert a Python type to a BOSS-style string.

Parameters:

valstr

Python object to stringify

File: pp/pp_main.py

boss.pp.pp_main.PPMain(bo_results, pp_iters=None, pp_acq_funcs=None, pp_models=None, pp_model_slice=None, pp_var_defaults=None, pp_truef_at_glmins=None, pp_truef_npts=None, pp_local_minima=None)

Performs the automated post-processing of a BOSS run.

File: pp/plot.py

Plotting functionality. All plot functions make PNG images to destination file based on given source array(s) of data.

boss.pp.plot.plot_acq_func(settings, dest_file, acqf_data, acqs=None, xhat=None, xnext=None, axis_labels=None, legends=True)

Plots a (max 2D) slice of the acquisition function. Can include also acquisitions, xhat and xnext if those are given.

boss.pp.plot.plot_conv_measures(settings, dest_file, conv_meas, legends=True)

Plots quantities related to convergence as a function of iteration.

boss.pp.plot.plot_data_acquisitions(settings, dest_file, acqs, min_preds)

Plots minimum predictions and uncertainty, acquisition energies and locations as a function of iteration.

boss.pp.plot.plot_hyperparameters(settings, dest_file, hypers, legends=True)

Plots GP model unfixed hyperparameters (variances and lengthscales) as a function of iteration.

boss.pp.plot.plot_model(settings, dest_file, model_data, xhat=None, acqs=None, xnext=None, minima=None, truef=None, incl_uncert=True, axis_labels=None, legends=True, paths=None)

Plots a (max 2D) slice of the model.

boss.pp.plot.plot_truef(settings, dest_file, truef_data, axis_labels=None)

Plots a (max 2D) slice of the true function.

boss.pp.plot.plot_truef_hat(settings, dest_file, truef_hats, legends=True)

Plots true function value at xhat locations as a function of iteration.

File: settings.py

class boss.settings.Settings(keywords, f=None)

Reads, interprets and defines the code internal settings based on input.

Attributes:

dim

The dimensionality of the user-supplied objective.

is_hsc

Indicates whether a heteroscedastic model is used.

is_multi

Indicates whether multi-task data is used.

Methods

check_acqfn()	Check the selected acquisition function.
check_multitask_settings()	Find inconsistencies and possible errors in multi-task settings.
clear()
correct()	Corrects the type and value of certain keywords.
dump(file_path[, only_user_keywords])	Writes the current settings to a boss input file.
from_file(file_path)	Factory method for Constructing a Settings object from a boss input file.
get(k[,d])
items()
keys()
pop(k[,d])	If key is not found, d is returned if given, otherwise KeyError is raised.
popitem()	as a 2-tuple; but raise KeyError if D is empty.
set_acqfn(name)	Will be deprecated.
set_dependent_defaults([only_missing])	Sets default values for keywords that depend on other keywords.
set_independent_defaults([only_missing])	Sets default values for independent keywords.
set_model(name)	Chooses model class based on settings.
setdefault(k[,d])
update([E, ]**F)	If E present and has a .keys() method, does: for k in E: D[k] = E[k] If E present and lacks .keys() method, does: for (k, v) in E: D[k] = v In either case, this is followed by: for k, v in F.items(): D[k] = v
values()

copy
deprecation_notice
fromkeys

__init__(keywords, f=None)

check_acqfn()

Check the selected acquisition function.

check_multitask_settings()

Find inconsistencies and possible errors in multi-task settings.

correct()

Corrects the type and value of certain keywords.

The user is afforded some laziness defining certain keywords, e.g., by providing lists instead of np.arrays.

property dim

The dimensionality of the user-supplied objective.

The number of dimensions is a read-only propery that is derived from the bounds provided by the user.

Returns:

int

The dimensionality of the objective.

dump(file_path, only_user_keywords=True)

Writes the current settings to a boss input file.

Parameters:

fnameUnion[str, path]

Path to the destination file.

classmethod from_file(file_path)

Factory method for Constructing a Settings object from a boss input file.

Parameters:

file_path: str, Path

Path to the input file.

Returns:

Settings

Settings object generated using the input file.

property is_hsc

Indicates whether a heteroscedastic model is used.

Returns:

bool

property is_multi

Indicates whether multi-task data is used.

Returns:

bool

set_acqfn(name)

Will be deprecated.

set_dependent_defaults(only_missing=True)

Sets default values for keywords that depend on other keywords.

set_independent_defaults(only_missing=True)

Sets default values for independent keywords.

set_model(name)

Chooses model class based on settings.

File: utils/distributions.py

boss.utils.distributions.fit_gumbel(mean, sd)

Helper function to obtain parameters for a Gumbel distribution. This distribution is used to model the distribution of the unknown global minimum.

The parameters are determined using mean and standard deviation values at random locations in the user function input space. With this, an empirical CDF P(Y_* < Y) can be constructed (probability of a random variable Y being larger than the (unknown) minimum Y_* of the user function, given what we know about the user function through the posterior model)

Mathematically, if Y_* is our (unknown) global minimum value, this empirical CDF can be constructed via

P(Y_* < Y) = 1 - prod_i^N CDF(-(Y-mu_i)/sigma_i)

if mu_i and sigma_i are the mean and standard deviations at N random locations from the user function input space. Rewrite this to stabilize the numeric computation:

P(Y_* < Y) = 1 - exp^{ln{sum_i^N CDF(-(Y-mu_i)/sigma_i)}}

We then solve this CDF for the 25th, 50th and 75th percentile and use the found Y to obtain Gumbel scaling parameters.

Parameters:

meanndarray

Mean values at num_random_samples values in the input domain

sdndarray

Corresponding standard deviations at the mean values

boss.utils.distributions.gammaparams(q1, q2, p1=0.1, p2=0.9)

A function for parametrizing gamma distributions by specifying two quantiles.

Relevant math can be found in “Determining distribution parameters from quantiles” by John D. Cook, 2010.

Parameters:

• q1 – p1-quantile

• q2 – p2-quantile

• p1 – lower percentage (0 < p1 < 1)

• p2 – higher percentage (0 < p2 < 1)

File: utils/minimization.py

class boss.utils.minimization.Minimization

Minimization utilities

Methods

minimize(func, bounds, kerntype, acqs, ...)	Tries to find global minimum of func by starting minimizers from accuracy*100 percentage of lowest acquisitions.
minimize_from_random(func, bounds, num_pts)	Tries to find global minimum of func by starting minimizers from num_pts random points (min 10 max 100).
minimize_from_sobol(func, bounds, num_pts[, ...])	Tries to find global minimum of func by starting minimizers from num_pts shifted sobol points.
minimize_using_score(func, bounds[, ...])	Searches the global minimum of func by starting minimizers from num_anchor points.
set_parallel(num_procs)	Set the number of parallel processes in the minimization step

static _minimize(arguments, has_gradient)

Wrapper on top of _run_minimizer for running either in parallel or in serial depending on whether a process pool has been created by Minimization.set_parallel().

static _remove_duplicates(original, min_distance)

Removes duplicates from a list of found local minima given a minimum distance between distinct minima.

static _run_minimizer(arguments)

Runs a single L-BFGS-B minimization starting from x0. Takes a single argument for easy compatibility with the multiprocessing library.

arguments is a tuple containing

func: The function to minimize. Must return value and gradients x0: Starting value bounds: Bounds passed to the minimizer args: arguments passed to the function func

static _run_minimizer_with_finite_gradients(arguments)

Runs a single L-BFGS-B minimization starting from x0 Approximate gradients using finite difference

func: The function to minimize. Must return value x0: Starting value bounds: Bounds passed to the minimizer args: arguments passed to the function func

static _within_bounds(x, bounds)

Check whether location x is within bounds.

static minimize(func, bounds, kerntype, acqs, min_dist_acqs, accuracy=0.3, args=(), lowest_min_only=True, has_gradient=True)

Tries to find global minimum of func by starting minimizers from accuracy*100 percentage of lowest acquisitions. func has to return both value and gradient given an x of same length as bounds.

static minimize_from_random(func, bounds, num_pts, args=(), lowest_min_only=True, has_gradient=True)

Tries to find global minimum of func by starting minimizers from num_pts random points (min 10 max 100). func has to return both value and gradient given an x of same length as bounds.

static minimize_from_sobol(func, bounds, num_pts, args=(), lowest_min_only=True, scramble=True, seed=None, has_gradient=True)

Tries to find global minimum of func by starting minimizers from num_pts shifted sobol points. func has to return both value and gradient given an x of same length as bounds.

static minimize_using_score(func, bounds, num_rnd_samples=10000, num_anchor=1, acqs=None, args=(), lowest_min_only=True, has_gradient=True)

Searches the global minimum of func by starting minimizers from num_anchor points. The points are chosen according to a score criteria, here the function itself is used to obtain the scores.

static set_parallel(num_procs)

Set the number of parallel processes in the minimization step

File: utils/timer.py

class boss.utils.timer.Timer

Utility class for wall clock time keeping.

Methods

getLapTime()	Returns lap time in seconds.
getTotalTime()	Returns total time since timer creation in seconds.
startLap()	Starts counting lap time from zero.
str_lapTime([units])	Returns lap time in the given unit of time as a string.
str_totalTime([units])	Returns total time since timer creation in the given unit of time as a string.

time

getLapTime()

Returns lap time in seconds.

getTotalTime()

Returns total time since timer creation in seconds.

startLap()

Starts counting lap time from zero.

str_lapTime(units='s')

Returns lap time in the given unit of time as a string.

str_totalTime(units='s')

Returns total time since timer creation in the given unit of time as a string.