How to specify bounds#
Constraints vs bounds#
optimagic distinguishes between bounds and constraints. Bounds are lower and upper bounds for parameters. In the literature, they are sometimes called box constraints. Examples for general constraints are linear constraints, probability constraints, or nonlinear constraints. You can find out more about general constraints in the next section on How to specify constraints.
Example objective function#
Let’s again look at the sphere function:
import numpy as np
import optimagic as om
def fun(x):
return x @ x
res = om.minimize(fun=fun, params=np.arange(3), algorithm="scipy_lbfgsb")
res.params.round(5)
array([ 0., -0., 0.])
Array params#
For params that are a numpy.ndarray
, one can specify the lower and/or upper-bounds as an array of the same length.
Lower bounds
res = om.minimize(
fun=fun,
params=np.arange(3),
bounds=om.Bounds(lower=np.ones(3)),
algorithm="scipy_lbfgsb",
)
res.params
array([1., 1., 1.])
Lower & upper-bounds
res = om.minimize(
fun=fun,
params=np.arange(3),
algorithm="scipy_lbfgsb",
bounds=om.Bounds(
lower=np.array([-2, -np.inf, 1]),
upper=np.array([-1, np.inf, np.inf]),
),
)
res.params
array([-1.00000000e+00, -3.57647466e-08, 1.00000000e+00])
Pytree params#
Now let’s look at a case where params is a more general pytree. We also update the sphere function by adding an intercept. Since the criterion always decreases when decreasing the intercept, there is no unrestricted solution. Lets fix a lower bound only for the intercept.
params = {"x": np.arange(3), "intercept": 3}
def fun(params):
return params["x"] @ params["x"] + params["intercept"]
res = om.minimize(
fun=fun,
params=params,
algorithm="scipy_lbfgsb",
bounds=om.Bounds(lower={"intercept": -2}),
)
res.params
{'x': array([ 0.00000000e+00, -4.42924006e-09, 2.04860640e-08]),
'intercept': np.float64(-2.0)}
optimagic tries to match the user provided bounds with the structure of params. This allows you to specify bounds for subtrees of params. In case your subtree specification results in an unidentified matching, optimagic will tell you so with a InvalidBoundsError
.
params data frame#
It often makes sense to specify your parameters in a pandas.DataFrame
, where you can utilize the multiindex for parameter naming. In this case, you can specify bounds as extra columns lower_bound
and upper_bound
.
Note The columns are called
*_bound
instead of*_bounds
like the argument passed tominimize
ormaximize
.
import pandas as pd
params = pd.DataFrame(
{"value": [0, 1, 2, 3], "lower_bound": [0, 1, 1, -2]},
index=pd.MultiIndex.from_tuples([("x", k) for k in range(3)] + [("intercept", 0)]),
)
params
value | lower_bound | ||
---|---|---|---|
x | 0 | 0 | 0 |
1 | 1 | 1 | |
2 | 2 | 1 | |
intercept | 0 | 3 | -2 |
def fun(params):
x = params.loc["x"]["value"].to_numpy()
intercept = params.loc["intercept"]["value"].iloc[0]
value = x @ x + intercept
return float(value)
res = om.minimize(
fun,
params=params,
algorithm="scipy_lbfgsb",
)
res.params
value | lower_bound | ||
---|---|---|---|
x | 0 | 0.0 | 0 |
1 | 1.0 | 1 | |
2 | 1.0 | 1 | |
intercept | 0 | -2.0 | -2 |
Coming from scipy#
If params
is a flat numpy array, you can also provide bounds in any format that
is supported by scipy.optimize.minimize
.