How to specify bounds#

Constraints vs bounds#

optimagic distinguishes between bounds and constraints. Bounds are lower and upper bounds for parameters. In the literature, they are sometimes called box constraints. Examples for general constraints are linear constraints, probability constraints, or nonlinear constraints. You can find out more about general constraints in the next section on How to specify constraints.

Example objective function#

Let’s again look at the sphere function:

import numpy as np
import optimagic as om

def fun(x):
    return x @ x

res = om.minimize(fun=fun, params=np.arange(3), algorithm="scipy_lbfgsb")
res.params.round(5)

array([ 0., -0.,  0.])

Array params#

For params that are a numpy.ndarray, one can specify the lower and/or upper-bounds as an array of the same length.

Lower bounds

res = om.minimize(
    fun=fun,
    params=np.arange(3),
    bounds=om.Bounds(lower=np.ones(3)),
    algorithm="scipy_lbfgsb",
)
res.params

array([1., 1., 1.])

Lower & upper-bounds

res = om.minimize(
    fun=fun,
    params=np.arange(3),
    algorithm="scipy_lbfgsb",
    bounds=om.Bounds(
        lower=np.array([-2, -np.inf, 1]),
        upper=np.array([-1, np.inf, np.inf]),
    ),
)
res.params

array([-1.00000000e+00, -3.57647466e-08,  1.00000000e+00])

Pytree params#

Now let’s look at a case where params is a more general pytree. We also update the sphere function by adding an intercept. Since the criterion always decreases when decreasing the intercept, there is no unrestricted solution. Lets fix a lower bound only for the intercept.

params = {"x": np.arange(3), "intercept": 3}


def fun(params):
    return params["x"] @ params["x"] + params["intercept"]

res = om.minimize(
    fun=fun,
    params=params,
    algorithm="scipy_lbfgsb",
    bounds=om.Bounds(lower={"intercept": -2}),
)
res.params

{'x': array([ 0.00000000e+00, -4.42924006e-09,  2.04860640e-08]),
 'intercept': -2.0}

optimagic tries to match the user provided bounds with the structure of params. This allows you to specify bounds for subtrees of params. In case your subtree specification results in an unidentified matching, optimagic will tell you so with a InvalidBoundsError.

params data frame#

It often makes sense to specify your parameters in a pandas.DataFrame, where you can utilize the multiindex for parameter naming. In this case, you can specify bounds as extra columns lower_bound and upper_bound.

Note The columns are called *_bound instead of *_bounds like the argument passed to minimize or maximize.

import pandas as pd

params = pd.DataFrame(
    {"value": [0, 1, 2, 3], "lower_bound": [0, 1, 1, -2]},
    index=pd.MultiIndex.from_tuples([("x", k) for k in range(3)] + [("intercept", 0)]),
)
params

		value	lower_bound
x	0	0	0
	1	1	1
	2	2	1
intercept	0	3	-2

def fun(params):
    x = params.loc["x"]["value"].to_numpy()
    intercept = params.loc["intercept"]["value"].iloc[0]
    value = x @ x + intercept
    return float(value)

res = om.minimize(
    fun,
    params=params,
    algorithm="scipy_lbfgsb",
)
res.params

		value	lower_bound
x	0	0.0	0
	1	1.0	1
	2	1.0	1
intercept	0	-2.0	-2

Coming from scipy#

If params is a flat numpy array, you can also provide bounds in any format that is supported by scipy.optimize.minimize.