API#

Optimization#

maximize
estimagic.maximize(criterion, params, algorithm, *, lower_bounds=None, upper_bounds=None, soft_lower_bounds=None, soft_upper_bounds=None, criterion_kwargs=None, constraints=None, algo_options=None, derivative=None, derivative_kwargs=None, criterion_and_derivative=None, criterion_and_derivative_kwargs=None, numdiff_options=None, logging=False, log_options=None, error_handling='raise', error_penalty=None, scaling=False, scaling_options=None, multistart=False, multistart_options=None, collect_history=True, skip_checks=False)[source]#

Maximize criterion using algorithm subject to constraints.

Parameters
  • criterion (callable) – A function that takes a params as first argument and returns a scalar (if only scalar algorithms will be used) or a dictionary that contains at the entries “value” (a scalar float), “contributions” (a pytree containing the summands that make up the criterion value) or “root_contributions” (a pytree containing the residuals of a least-squares problem) and any number of additional entries. The additional dict entries will be stored in a database if logging is used.

  • params (pandas) – A pytree containing the parameters with respect to which the criterion is optimized. Examples are a numpy array, a pandas Series, a DataFrame with “value” column, a float and any kind of (nested) dictionary or list containing these elements. See How to specify params for examples.

  • algorithm (str or callable) – Specifies the optimization algorithm. For built-in algorithms this is a string with the name of the algorithm. Otherwise it can be a callable with the estimagic algorithm interface. See How to specify algorithms and algorithm specific options.

  • lower_bounds (pytree) – A pytree with the same structure as params with lower bounds for the parameters. Can be -np.inf for parameters with no lower bound.

  • upper_bounds (pytree) – As lower_bounds. Can be np.inf for parameters with no upper bound.

  • soft_lower_bounds (pytree) – As lower bounds but the bounds are not imposed during optimization and just used to sample start values if multistart optimization is performed.

  • soft_upper_bounds (pytree) – As soft_lower_bounds.

  • criterion_kwargs (dict) – Additional keyword arguments for criterion

  • constraints (list, dict) – List with constraint dictionaries or single dict. See How to specify constraints.

  • algo_options (dict) – Algorithm specific configuration of the optimization. See Optimizers for supported options of each algorithm.

  • derivative (callable) – Function that calculates the first derivative of criterion. For most algorithm, this is the gradient of the scalar output (or “value” entry of the dict). However some algorithms (e.g. bhhh) require the jacobian of the “contributions” entry of the dict. You will get an error if you provide the wrong type of derivative.

  • derivative_kwargs (dict) – Additional keyword arguments for derivative.

  • criterion_and_derivative (callable) – Function that returns criterion and derivative as a tuple. This can be used to exploit synergies in the evaluation of both functions. The first element of the tuple has to be exactly the same as the output of criterion. The second has to be exactly the same as the output of derivative.

  • criterion_and_derivative_kwargs (dict) – Additional keyword arguments for criterion and derivative.

  • numdiff_options (dict) – Keyword arguments for the calculation of numerical derivatives. See Derivatives for details. Note that the default method is changed to “forward” for speed reasons.

  • logging (pathlib.Path, str or False) – Path to sqlite3 file (which typically has the file extension .db. If the file does not exist, it will be created. When doing parallel optimizations and logging is provided, you have to provide a different path for each optimization you are running. You can disable logging completely by setting it to False, but we highly recommend not to do so. The dashboard can only be used when logging is used.

  • log_options (dict) – Additional keyword arguments to configure the logging. - “fast_logging”: A boolean that determines if “unsafe” settings are used to speed up write processes to the database. This should only be used for very short running criterion functions where the main purpose of the log is a real-time dashboard and it would not be catastrophic to get a corrupted database in case of a sudden system shutdown. If one evaluation of the criterion function (and gradient if applicable) takes more than 100 ms, the logging overhead is negligible. - “if_table_exists”: (str) One of “extend”, “replace”, “raise”. What to do if the tables we want to write to already exist. Default “extend”. - “if_database_exists”: (str): One of “extend”, “replace”, “raise”. What to do if the database we want to write to already exists. Default “extend”.

  • error_handling (str) – Either “raise” or “continue”. Note that “continue” does not absolutely guarantee that no error is raised but we try to handle as many errors as possible in that case without aborting the optimization.

  • error_penalty (dict) – Dict with the entries “constant” (float) and “slope” (float). If the criterion or gradient raise an error and error_handling is “continue”, return constant + slope * norm(params - start_params) where norm is the euclidean distance as criterion value and adjust the derivative accordingly. This is meant to guide the optimizer back into a valid region of parameter space (in direction of the start parameters). Note that the constant has to be high enough to ensure that the penalty is actually a bad function value. The default constant is f0 + abs(f0) + 100 for minimizations and f0 - abs(f0) - 100 for maximizations, where f0 is the criterion value at start parameters. The default slope is 0.1.

  • scaling (bool) – If True, the parameter vector is rescaled internally for better performance with scale sensitive optimizers.

  • scaling_options (dict or None) – Options to configure the internal scaling ot the parameter vector. See How to scale optimization problems for details and recommendations.

  • multistart (bool) – Whether to do the optimization from multiple starting points. Requires the params to have the columns "soft_lower_bound" and "soft_upper_bounds" with finite values for all parameters, unless the standard bounds are already finite for all parameters.

  • multistart_options (dict) – Options to configure the optimization from multiple starting values. The dictionary has the following entries (all of which are optional): - n_samples (int): Number of sampled points on which to do one function evaluation. Default is 10 * n_params. - sample (pandas.DataFrame or numpy.ndarray) A user definde sample. If this is provided, n_samples, sampling_method and sampling_distribution are not used. - share_optimizations (float): Share of sampled points that is used to construct a starting point for a local optimization. Default 0.1. - sampling_distribution (str): One rof “uniform”, “triangle”. Default is “uniform” as in the original tiktak algorithm. - sampling_method (str): One of “random”, “sobol”, “halton”, “hammersley”, “korobov”, “latin_hypercube” or a numpy array or DataFrame with custom points. Default is sobol for problems with up to 30 parameters and random for problems with more than 30 parameters. - mixing_weight_method (str or callable): Specifies how much weight is put on the currently best point when calculating a new starting point for a local optimization out of the currently best point and the next random starting point. Either “tiktak” or “linear” or a callable that takes the arguments iteration, n_iterations, min_weight, max_weight. Default “tiktak”. - mixing_weight_bounds (tuple): A tuple consisting of a lower and upper bound on mixing weights. Default (0.1, 0.995). - convergence_max_discoveries (int): The multistart optimization converges if the currently best local optimum has been discovered independently in convergence_max_discoveries many local optimizations. Default 2. - convergence.relative_params_tolerance (float): Determines the maximum relative distance two parameter vectors can have to be considered equal for convergence purposes. - n_cores (int): Number cores used to evaluate the criterion function in parallel during exploration stages and number of parallel local optimization in optimization stages. Default 1. - batch_evaluator (str or callable): See Batch evaluators for details. Default “joblib”. - batch_size (int): If n_cores is larger than one, several starting points for local optimizations are created with the same weight and from the same currently best point. The batch_size argument is a way to reproduce this behavior on a small machine where less cores are available. By default the batch_size is equal to n_cores. It can never be smaller than n_cores. - seed (int): Random seed for the creation of starting values. Default None. - exploration_error_handling (str): One of “raise” or “continue”. Default is continue, which means that failed function evaluations are simply discarded from the sample. - optimization_error_handling (str): One of “raise” or “continue”. Default is continue, which means that failed optimizations are simply discarded.

  • collect_history (bool) – Whether the history of parameters and criterion values should be collected and returned as part of the result. Default True.

  • skip_checks (bool) – Whether checks on the inputs are skipped. This makes the optimization faster, especially for very fast criterion functions. Default False.

Returns

The optmization result.

Return type

OptimizeResult

minimize
estimagic.minimize(criterion, params, algorithm, *, lower_bounds=None, upper_bounds=None, soft_lower_bounds=None, soft_upper_bounds=None, criterion_kwargs=None, constraints=None, algo_options=None, derivative=None, derivative_kwargs=None, criterion_and_derivative=None, criterion_and_derivative_kwargs=None, numdiff_options=None, logging=False, log_options=None, error_handling='raise', error_penalty=None, scaling=False, scaling_options=None, multistart=False, multistart_options=None, collect_history=True, skip_checks=False)[source]#

Minimize criterion using algorithm subject to constraints.

Parameters
  • criterion (callable) – A function that takes a params as first argument and returns a scalar (if only scalar algorithms will be used) or a dictionary that contains at the entries “value” (a scalar float), “contributions” (a pytree containing the summands that make up the criterion value) or “root_contributions” (a pytree containing the residuals of a least-squares problem) and any number of additional entries. The additional dict entries will be stored in a database if logging is used.

  • params (pandas) – A pytree containing the parameters with respect to which the criterion is optimized. Examples are a numpy array, a pandas Series, a DataFrame with “value” column, a float and any kind of (nested) dictionary or list containing these elements. See How to specify params for examples.

  • algorithm (str or callable) – Specifies the optimization algorithm. For built-in algorithms this is a string with the name of the algorithm. Otherwise it can be a callable with the estimagic algorithm interface. See How to specify algorithms and algorithm specific options.

  • lower_bounds (pytree) – A pytree with the same structure as params with lower bounds for the parameters. Can be -np.inf for parameters with no lower bound.

  • upper_bounds (pytree) – As lower_bounds. Can be np.inf for parameters with no upper bound.

  • soft_lower_bounds (pytree) – As lower bounds but the bounds are not imposed during optimization and just used to sample start values if multistart optimization is performed.

  • soft_upper_bounds (pytree) – As soft_lower_bounds.

  • criterion_kwargs (dict) – Additional keyword arguments for criterion

  • constraints (list, dict) – List with constraint dictionaries or single dict. See How to specify constraints.

  • algo_options (dict) – Algorithm specific configuration of the optimization. See Optimizers for supported options of each algorithm.

  • derivative (callable) – Function that calculates the first derivative of criterion. For most algorithm, this is the gradient of the scalar output (or “value” entry of the dict). However some algorithms (e.g. bhhh) require the jacobian of the “contributions” entry of the dict. You will get an error if you provide the wrong type of derivative.

  • derivative_kwargs (dict) – Additional keyword arguments for derivative.

  • criterion_and_derivative (callable) – Function that returns criterion and derivative as a tuple. This can be used to exploit synergies in the evaluation of both functions. The first element of the tuple has to be exactly the same as the output of criterion. The second has to be exactly the same as the output of derivative.

  • criterion_and_derivative_kwargs (dict) – Additional keyword arguments for criterion and derivative.

  • numdiff_options (dict) – Keyword arguments for the calculation of numerical derivatives. See Derivatives for details. Note that the default method is changed to “forward” for speed reasons.

  • logging (pathlib.Path, str or False) – Path to sqlite3 file (which typically has the file extension .db. If the file does not exist, it will be created. When doing parallel optimizations and logging is provided, you have to provide a different path for each optimization you are running. You can disable logging completely by setting it to False, but we highly recommend not to do so. The dashboard can only be used when logging is used.

  • log_options (dict) – Additional keyword arguments to configure the logging. - “fast_logging”: A boolean that determines if “unsafe” settings are used to speed up write processes to the database. This should only be used for very short running criterion functions where the main purpose of the log is a real-time dashboard and it would not be catastrophic to get a corrupted database in case of a sudden system shutdown. If one evaluation of the criterion function (and gradient if applicable) takes more than 100 ms, the logging overhead is negligible. - “if_table_exists”: (str) One of “extend”, “replace”, “raise”. What to do if the tables we want to write to already exist. Default “extend”. - “if_database_exists”: (str): One of “extend”, “replace”, “raise”. What to do if the database we want to write to already exists. Default “extend”.

  • error_handling (str) – Either “raise” or “continue”. Note that “continue” does not absolutely guarantee that no error is raised but we try to handle as many errors as possible in that case without aborting the optimization.

  • error_penalty (dict) – Dict with the entries “constant” (float) and “slope” (float). If the criterion or gradient raise an error and error_handling is “continue”, return constant + slope * norm(params - start_params) where norm is the euclidean distance as criterion value and adjust the derivative accordingly. This is meant to guide the optimizer back into a valid region of parameter space (in direction of the start parameters). Note that the constant has to be high enough to ensure that the penalty is actually a bad function value. The default constant is f0 + abs(f0) + 100 for minimizations and f0 - abs(f0) - 100 for maximizations, where f0 is the criterion value at start parameters. The default slope is 0.1.

  • scaling (bool) – If True, the parameter vector is rescaled internally for better performance with scale sensitive optimizers.

  • scaling_options (dict or None) – Options to configure the internal scaling ot the parameter vector. See How to scale optimization problems for details and recommendations.

  • multistart (bool) – Whether to do the optimization from multiple starting points. Requires the params to have the columns "soft_lower_bound" and "soft_upper_bounds" with finite values for all parameters, unless the standard bounds are already finite for all parameters.

  • multistart_options (dict) – Options to configure the optimization from multiple starting values. The dictionary has the following entries (all of which are optional): - n_samples (int): Number of sampled points on which to do one function evaluation. Default is 10 * n_params. - sample (pandas.DataFrame or numpy.ndarray) A user definde sample. If this is provided, n_samples, sampling_method and sampling_distribution are not used. - share_optimizations (float): Share of sampled points that is used to construct a starting point for a local optimization. Default 0.1. - sampling_distribution (str): One rof “uniform”, “triangle”. Default is “uniform” as in the original tiktak algorithm. - sampling_method (str): One of “random”, “sobol”, “halton”, “hammersley”, “korobov”, “latin_hypercube” or a numpy array or DataFrame with custom points. Default is sobol for problems with up to 30 parameters and random for problems with more than 30 parameters. - mixing_weight_method (str or callable): Specifies how much weight is put on the currently best point when calculating a new starting point for a local optimization out of the currently best point and the next random starting point. Either “tiktak” or “linear” or a callable that takes the arguments iteration, n_iterations, min_weight, max_weight. Default “tiktak”. - mixing_weight_bounds (tuple): A tuple consisting of a lower and upper bound on mixing weights. Default (0.1, 0.995). - convergence_max_discoveries (int): The multistart optimization converges if the currently best local optimum has been discovered independently in convergence_max_discoveries many local optimizations. Default 2. - convergence.relative_params_tolerance (float): Determines the maximum relative distance two parameter vectors can have to be considered equal for convergence purposes. - n_cores (int): Number cores used to evaluate the criterion function in parallel during exploration stages and number of parallel local optimization in optimization stages. Default 1. - batch_evaluator (str or callable): See Batch evaluators for details. Default “joblib”. - batch_size (int): If n_cores is larger than one, several starting points for local optimizations are created with the same weight and from the same currently best point. The batch_size argument is a way to reproduce this behavior on a small machine where less cores are available. By default the batch_size is equal to n_cores. It can never be smaller than n_cores. - seed (int): Random seed for the creation of starting values. Default None. - exploration_error_handling (str): One of “raise” or “continue”. Default is continue, which means that failed function evaluations are simply discarded from the sample. - optimization_error_handling (str): One of “raise” or “continue”. Default is continue, which means that failed optimizations are simply discarded.

  • collect_history (bool) – Whether the history of parameters and criterion values should be collected and returned as part of the result. Default True.

  • skip_checks (bool) – Whether checks on the inputs are skipped. This makes the optimization faster, especially for very fast criterion functions. Default False.

Returns

The optmization result.

Return type

OptimizeResult

slice_plot
estimagic.slice_plot(func, params, lower_bounds=None, upper_bounds=None, func_kwargs=None, selector=None, n_cores=1, n_gridpoints=20, plots_per_row=2, param_names=None, share_y=True, expand_yrange=0.02, share_x=False, color='#497ea7', template='simple_white', title=None, return_dict=False, make_subplot_kwargs=None, batch_evaluator='joblib')[source]#

Plot criterion along coordinates at given and random values.

Generates plots for each parameter and optionally combines them into a figure with subplots.

Parameters
  • criterion (callable) – criterion function that takes params and returns a scalar value or dictionary with the entry “value”.

  • params (pytree) – A pytree with parameters.

  • lower_bounds (pytree) – A pytree with same structure as params. Must be specified and finite for all parameters unless params is a DataFrame containing with “lower_bound” column.

  • upper_bounds (pytree) – A pytree with same structure as params. Must be specified and finite for all parameters unless params is a DataFrame containing with “lower_bound” column.

  • selector (callable) – Function that takes params and returns a subset of params for which we actually want to generate the plot.

  • n_cores (int) – Number of cores.

  • n_gridpoins (int) – Number of gridpoints on which the criterion function is evaluated. This is the number per plotted line.

  • plots_per_row (int) – Number of plots per row.

  • param_names (dict or NoneType) – Dictionary mapping old parameter names to new ones.

  • share_y (bool) – If True, the individual plots share the scale on the yaxis and plots in one row actually share the y axis.

  • share_x (bool) – If True, set the same range of x axis for all plots and share the x axis for all plots in one column.

  • expand_y (float) – The ration by which to expand the range of the (shared) y axis, such that the axis is not cropped at exactly max of Criterion Value.

  • color – The line color.

  • template (str) – The template for the figure. Default is “plotly_white”.

  • layout_kwargs (dict or NoneType) – Dictionary of key word arguments used to update layout of plotly Figure object. If None, the default kwargs defined in the function will be used.

  • title (str) – The figure title.

  • return_dict (bool) – If True, return dictionary with individual plots of each parameter, else, ombine individual plots into a figure with subplots.

  • make_subplot_kwargs (dict or NoneType) – Dictionary of keyword arguments used to instantiate plotly Figure with multiple subplots. Is used to define properties such as, for example, the spacing between subplots (governed by ‘horizontal_spacing’ and ‘vertical_spacing’). If None, default arguments defined in the function are used.

  • batch_evaluator (str or callable) – See Batch evaluators.

Returns

Returns either dictionary with individual slice

plots for each parameter or a plotly Figure combining the individual plots.

Return type

out (dict or plotly.Figure)

criterion_plot
estimagic.criterion_plot(results, names=None, max_evaluations=None, template='simple_white', palette=['rgb(102,194,165)', 'rgb(252,141,98)', 'rgb(141,160,203)', 'rgb(231,138,195)', 'rgb(166,216,84)', 'rgb(255,217,47)', 'rgb(229,196,148)', 'rgb(179,179,179)'], stack_multistart=False, monotone=False, show_exploration=False)[source]#

Plot the criterion history of an optimization.

Parameters
  • results (Union[List, Dict][Union[OptimizeResult, pathlib.Path, str]) – A (list or dict of) optimization results with collected history. If dict, then the key is used as the name in a legend.

  • names (Union[List[str], str]) – Names corresponding to res or entries in res.

  • max_evaluations (int) – Clip the criterion history after that many entries.

  • template (str) – The template for the figure. Default is “plotly_white”.

  • palette (Union[List[str], str]) – The coloring palette for traces. Default is “qualitative.Plotly”.

  • stack_multistart (bool) – Whether to combine multistart histories into a single history. Default is False.

  • monotone (bool) – If True, the criterion plot becomes monotone in the sense that only that at each iteration the current best criterion value is displayed. Default is False.

  • show_exploration (bool) – If True, exploration samples of a multistart optimization are visualized. Default is False.

Returns

The figure.

Return type

plotly.graph_objs._figure.Figure

params_plot
estimagic.params_plot(result, selector=None, max_evaluations=None, template='simple_white', show_exploration=False)[source]#

Plot the params history of an optimization.

Parameters
  • result (Union[OptimizeResult, pathlib.Path, str]) – An optimization results with collected history. If dict, then the key is used as the name in a legend.

  • selector (callable) – A callable that takes params and returns a subset of params. If provided, only the selected subset of params is plotted.

  • max_evaluations (int) – Clip the criterion history after that many entries.

  • template (str) – The template for the figure. Default is “plotly_white”.

  • show_exploration (bool) – If True, exploration samples of a multistart optimization are visualized. Default is False.

Returns

The figure.

Return type

plotly.graph_objs._figure.Figure

count_free_params
estimagic.count_free_params(params, constraints=None, lower_bounds=None, upper_bounds=None)[source]#

Count the (free) parameters of an optimization problem.

Parameters
  • params (pytree) – The parameters.

  • constraints (list) – The constraints for the optimization problem. If constraints are provided, only the free parameters are counted.

  • lower_bounds (pytree) – Lower bounds for params.

  • upper_bounds (pytree) – Upper bounds for params.

Returns

Number of (free) parameters

Return type

int

check_constraints
estimagic.check_constraints(params, constraints, lower_bounds=None, upper_bounds=None)[source]#

Raise an error if constraints are invalid or not satisfied in params.

Parameters
  • params (pytree) – The parameters.

  • constraints (list) – The constraints for the optimization problem.

  • lower_bounds (pytree) – Lower bounds for params.

  • upper_bounds (pytree) – Upper bounds for params.

Raises
  • InvalidParamsError – If constraints are valid but not satisfied.

  • InvalidConstraintError – If constraints are invalid.

OptimizeResult
class estimagic.OptimizeResult(params: typing.Any, criterion: float, start_criterion: float, start_params: typing.Any, algorithm: str, direction: str, n_free: int, message: typing.Optional[str] = None, success: typing.Optional[bool] = None, n_criterion_evaluations: typing.Optional[int] = None, n_derivative_evaluations: typing.Optional[int] = None, n_iterations: typing.Optional[int] = None, history: typing.Optional[typing.Dict] = None, convergence_report: typing.Optional[typing.Dict] = None, multistart_info: typing.Optional[typing.Dict] = None, algorithm_output: typing.Dict = <factory>)[source]#

Optimization result object.

Attributes

params#

The optimal parameters.

Type

Any

criterion#

The optimal criterion value.

Type

float

start_criterion#

The criterion value at the start parameters.

Type

float

start_params#

The start parameters.

Type

Any

algorithm#

The algorithm used for the optimization.

Type

str

direction#

Maximize or minimize.

Type

str

n_free#

Number of free parameters.

Type

int

message#

Message returned by the underlying algorithm.

Type

Union[str, None] = None

success#

Whether the optimization was successful.

Type

Union[bool, None] = None

n_criterion_evaluations#

Number of criterion evaluations.

Type

Union[int, None] = None

n_derivative_evaluations#

Number of derivative evaluations.

Type

Union[int, None] = None

n_iterations#

Number of iterations until termination.

Type

Union[int, None] = None

history#

Optimization history.

Type

Union[Dict, None] = None

convergence_report#

The convergence report.

Type

Union[Dict, None] = None

multistart_info#

Multistart information.

Type

Union[Dict, None] = None

algorithm_output#

Additional algorithm specific information.

Type

Dict = field(default_factory=dict)

to_pickle(path)[source]#

Save the OptimizeResult object to pickle.

Parameters

path (str, pathlib.Path) – A str or pathlib.path ending in .pkl or .pickle.

Derivatives#

first_derivative
estimagic.first_derivative(func, params, *, func_kwargs=None, method='central', n_steps=1, base_steps=None, scaling_factor=1, lower_bounds=None, upper_bounds=None, step_ratio=2, min_steps=None, f0=None, n_cores=1, error_handling='continue', batch_evaluator='joblib', return_func_value=False, return_info=False, key=None)[source]#

Evaluate first derivative of func at params according to method and step options.

Internally, the function is converted such that it maps from a 1d array to a 1d array. Then the Jacobian of that function is calculated.

The parameters and the function output can be estimagic-pytrees; for more details on estimagi-pytrees see EEP-01: Pytrees. By default the resulting Jacobian will be returned as a block-pytree.

For a detailed description of all options that influence the step size as well as an explanation of how steps are adjusted to bounds in case of a conflict, see generate_steps().

Parameters
  • func (callable) – Function of which the derivative is calculated.

  • params (pytree) – A pytree. See How to specify params.

  • func_kwargs (dict) – Additional keyword arguments for func, optional.

  • method (str) – One of [“central”, “forward”, “backward”], default “central”.

  • n_steps (int) – Number of steps needed. For central methods, this is the number of steps per direction. It is 1 if no Richardson extrapolation is used.

  • base_steps (numpy.ndarray, optional) – 1d array of the same length as params. base_steps * scaling_factor is the absolute value of the first (and possibly only) step used in the finite differences approximation of the derivative. If base_steps * scaling_factor conflicts with bounds, the actual steps will be adjusted. If base_steps is not provided, it will be determined according to a rule of thumb as long as this does not conflict with min_steps.

  • scaling_factor (numpy.ndarray or float) – Scaling factor which is applied to base_steps. If it is an numpy.ndarray, it needs to be as long as params. scaling_factor is useful if you want to increase or decrease the base_step relative to the rule-of-thumb or user provided base_step, for example to benchmark the effect of the step size. Default 1.

  • lower_bounds (pytree) – To be written.

  • upper_bounds (pytree) – To be written.

  • step_ratio (float, numpy.array) – Ratio between two consecutive Richardson extrapolation steps in the same direction. default 2.0. Has to be larger than one. The step ratio is only used if n_steps > 1.

  • min_steps (numpy.ndarray) – Minimal possible step sizes that can be chosen to accommodate bounds. Must have same length as params. By default min_steps is equal to base_steps, i.e step size is not decreased beyond what is optimal according to the rule of thumb.

  • f0 (numpy.ndarray) – 1d numpy array with func(x), optional.

  • n_cores (int) – Number of processes used to parallelize the function evaluations. Default 1.

  • error_handling (str) – One of “continue” (catch errors and continue to calculate derivative estimates. In this case, some derivative estimates can be missing but no errors are raised), “raise” (catch errors and continue to calculate derivative estimates at first but raise an error if all evaluations for one parameter failed) and “raise_strict” (raise an error as soon as a function evaluation fails).

  • batch_evaluator (str or callable) – Name of a pre-implemented batch evaluator (currently ‘joblib’ and ‘pathos_mp’) or Callable with the same interface as the estimagic batch_evaluators.

  • return_func_value (bool) – If True, return function value at params, stored in output dict under “func_value”. Default False. This is useful when using first_derivative during optimization.

  • return_info (bool) – If True, return additional information on function evaluations and internal derivative candidates, stored in output dict under “func_evals” and “derivative_candidates”. Derivative candidates are only returned if n_steps > 1. Default False.

  • key (str) – If func returns a dictionary, take the derivative of func(params)[key].

Returns

Result dictionary with keys:
  • ”derivative” (numpy.ndarray, pandas.Series or pandas.DataFrame): The

    estimated first derivative of func at params. The shape of the output depends on the dimension of params and func(params):

    • f: R -> R leads to shape (1,), usually called derivative

    • f: R^m -> R leads to shape (m, ), usually called Gradient

    • f: R -> R^n leads to shape (n, 1), usually called Jacobian

    • f: R^m -> R^n leads to shape (n, m), usually called Jacobian

  • ”func_value” (numpy.ndarray, pandas.Series or pandas.DataFrame): Function

    value at params, returned if return_func_value is True.

  • ”func_evals” (pandas.DataFrame): Function evaluations produced by internal

    derivative method, returned if return_info is True.

  • ”derivative_candidates” (pandas.DataFrame): Derivative candidates from

    Richardson extrapolation, returned if return_info is True and n_steps > 1.

Return type

result (dict)

second_derivative
estimagic.second_derivative(func, params, *, func_kwargs=None, method='central_cross', n_steps=1, base_steps=None, scaling_factor=1, lower_bounds=None, upper_bounds=None, step_ratio=2, min_steps=None, f0=None, n_cores=1, error_handling='continue', batch_evaluator='joblib', return_func_value=False, return_info=False, key=None)[source]#

Evaluate second derivative of func at params according to method and step options.

Internally, the function is converted such that it maps from a 1d array to a 1d array. Then the Hessians of that function are calculated. The resulting derivative estimate is always a numpy.ndarray.

The parameters and the function output can be pandas objects (Series or DataFrames with value column). In that case the output of second_derivative is also a pandas object and with appropriate index and columns.

Detailed description of all options that influence the step size as well as an explanation of how steps are adjusted to bounds in case of a conflict, see generate_steps().

Parameters
  • func (callable) – Function of which the derivative is calculated.

  • params (numpy.ndarray, pandas.Series or pandas.DataFrame) – 1d numpy array or pandas.DataFrame with parameters at which the derivative is calculated. If it is a DataFrame, it can contain the columns “lower_bound” and “upper_bound” for bounds. See How to specify params.

  • func_kwargs (dict) – Additional keyword arguments for func, optional.

  • method (str) – One of {“forward”, “backward”, “central_average”, “central_cross”} These correspond to the finite difference approximations defined in equations [7, x, 8, 9] in Rideout [2009], where (“backward”, x) is not found in Rideout [2009] but is the natural extension of equation 7 to the backward case. Default “central_cross”.

  • n_steps (int) – Number of steps needed. For central methods, this is the number of steps per direction. It is 1 if no Richardson extrapolation is used.

  • base_steps (numpy.ndarray, optional) – 1d array of the same length as params. base_steps * scaling_factor is the absolute value of the first (and possibly only) step used in the finite differences approximation of the derivative. If base_steps * scaling_factor conflicts with bounds, the actual steps will be adjusted. If base_steps is not provided, it will be determined according to a rule of thumb as long as this does not conflict with min_steps.

  • scaling_factor (numpy.ndarray or float) – Scaling factor which is applied to base_steps. If it is an numpy.ndarray, it needs to be as long as params. scaling_factor is useful if you want to increase or decrease the base_step relative to the rule-of-thumb or user provided base_step, for example to benchmark the effect of the step size. Default 1.

  • lower_bounds (numpy.ndarray) – 1d array with lower bounds for each parameter. If params is a DataFrame and has the columns “lower_bound”, this will be taken as lower_bounds if now lower_bounds have been provided explicitly.

  • upper_bounds (numpy.ndarray) – 1d array with upper bounds for each parameter. If params is a DataFrame and has the columns “upper_bound”, this will be taken as upper_bounds if no upper_bounds have been provided explicitly.

  • step_ratio (float, numpy.array) – Ratio between two consecutive Richardson extrapolation steps in the same direction. default 2.0. Has to be larger than one. The step ratio is only used if n_steps > 1.

  • min_steps (numpy.ndarray) – Minimal possible step sizes that can be chosen to accommodate bounds. Must have same length as params. By default min_steps is equal to base_steps, i.e step size is not decreased beyond what is optimal according to the rule of thumb.

  • f0 (numpy.ndarray) – 1d numpy array with func(x), optional.

  • n_cores (int) – Number of processes used to parallelize the function evaluations. Default 1.

  • error_handling (str) – One of “continue” (catch errors and continue to calculate derivative estimates. In this case, some derivative estimates can be missing but no errors are raised), “raise” (catch errors and continue to calculate derivative estimates at first but raise an error if all evaluations for one parameter failed) and “raise_strict” (raise an error as soon as a function evaluation fails).

  • batch_evaluator (str or callable) – Name of a pre-implemented batch evaluator (currently ‘joblib’ and ‘pathos_mp’) or Callable with the same interface as the estimagic batch_evaluators.

  • return_func_value (bool) – If True, return function value at params, stored in output dict under “func_value”. Default False. This is useful when using first_derivative during optimization.

  • return_info (bool) – If True, return additional information on function evaluations and internal derivative candidates, stored in output dict under “func_evals” and “derivative_candidates”. Derivative candidates are only returned if n_steps > 1. Default False.

  • key (str) – If func returns a dictionary, take the derivative of func(params)[key].

Returns

Result dictionary with keys:
  • ”derivative” (numpy.ndarray, pandas.Series or pandas.DataFrame): The

    estimated second derivative of func at params. The shape of the output depends on the dimension of params and func(params):

    • f: R -> R leads to shape (1,), usually called second derivative

    • f: R^m -> R leads to shape (m, m), usually called Hessian

    • f: R -> R^n leads to shape (n,), usually called Hessian

    • f: R^m -> R^n leads to shape (n, m, m), usually called Hessian tensor

  • ”func_value” (numpy.ndarray, pandas.Series or pandas.DataFrame): Function

    value at params, returned if return_func_value is True.

  • ”func_evals_one_step” (pandas.DataFrame): Function evaluations produced by

    internal derivative method when altering the params vector at one dimension, returned if return_info is True.

  • ”func_evals_two_step” (pandas.DataFrame): This features is not implemented

    yet and therefore set to None. Once implemented it will contain function evaluations produced by internal derivative method when altering the params vector at two dimensions, returned if return_info is True.

  • ”func_evals_cross_step” (pandas.DataFrame): This features is not

    implemented yet and therefore set to None. Once implemented it will contain function evaluations produced by internal derivative method when altering the params vector at two dimensions in different directions, returned if return_info is True.

Return type

result (dict)

derivative_plot
estimagic.derivative_plot(derivative_result, combine_plots_in_grid=True, template='simple_white', palette=['rgb(102,194,165)', 'rgb(252,141,98)', 'rgb(141,160,203)', 'rgb(231,138,195)', 'rgb(166,216,84)', 'rgb(255,217,47)', 'rgb(229,196,148)', 'rgb(179,179,179)'])[source]#

Plot evaluations and derivative estimates.

The resulting grid plot displays function evaluations and derivatives. The derivatives are visualized as a first-order Taylor approximation. Bands are drawn indicating the area in which forward and backward derivatives are located. This is done by filling the area between the derivative estimate with lowest and highest step size, respectively. Do not confuse these bands with statistical errors.

This function does not require the params vector as plots are displayed relative to the point at which the derivative is calculated.

Parameters
  • derivative_result (dict) – The result dictionary of call to first_derivative() with return_info and return_func_value set to True.

  • combine_plots_in_grid (bool) – decide whether to return a one figure containing subplots for each factor pair or a dictionary of individual plots. Default True.

  • template (str) – The template for the figure. Default is “plotly_white”.

  • palette – The coloring palette for traces. Default is “qualitative.Plotly”.

Returns

The grid plot or dict of individual plots

Return type

plotly.Figure

Estimation#

estimate_ml
estimagic.estimate_ml(loglike, params, optimize_options, *, lower_bounds=None, upper_bounds=None, constraints=None, logging=False, log_options=None, loglike_kwargs=None, numdiff_options=None, jacobian=None, jacobian_kwargs=None, hessian=None, hessian_kwargs=None, design_info=None)[source]#

Do a maximum likelihood (ml) estimation.

This is a high level interface of our lower level functions for maximization, numerical differentiation and inference. It does the full workflow for maximum likelihood estimation with just one function call.

While we have good defaults, you can still configure each aspect of each step via the optional arguments of this function. If you find it easier to do the maximization separately, you can do so and just provide the optimal parameters as params and set optimize_options=False

Parameters
  • loglike (callable) – Likelihood function that takes a params (and potentially other keyword arguments) and returns a dictionary that has at least the entries “value” (a scalar float) and “contributions” (a 1d numpy array or pytree) with the log likelihood contribution per individual.

  • params (pytree) – A pytree containing the estimated or start parameters of the likelihood model. If the supplied parameters are estimated parameters, set optimize_options to False. Pytrees can be a numpy array, a pandas Series, a DataFrame with “value” column, a float and any kind of (nested) dictionary or list containing these elements. See How to specify params for examples.

  • optimize_options (dict, str or False) – Keyword arguments that govern the numerical optimization. Valid entries are all arguments of minimize() except for those that are passed explicilty to estimate_ml. If you pass False as optimize_options you signal that params are already the optimal parameters and no numerical optimization is needed. If you pass a str as optimize_options it is used as the algorithm option.

  • lower_bounds (pytree) – A pytree with the same structure as params with lower bounds for the parameters. Can be -np.inf for parameters with no lower bound.

  • upper_bounds (pytree) – As lower_bounds. Can be np.inf for parameters with no upper bound.

  • constraints (list, dict) – List with constraint dictionaries or single dict. See How to specify constraints.

  • logging (pathlib.Path, str or False) – Path to sqlite3 file (which typically has the file extension .db. If the file does not exist, it will be created. The dashboard can only be used when logging is used.

  • log_options (dict) – Additional keyword arguments to configure the logging. - “fast_logging”: A boolean that determines if “unsafe” settings are used to speed up write processes to the database. This should only be used for very short running criterion functions where the main purpose of the log is a real-time dashboard and it would not be catastrophic to get a corrupted database in case of a sudden system shutdown. If one evaluation of the criterion function (and gradient if applicable) takes more than 100 ms, the logging overhead is negligible. - “if_table_exists”: (str) One of “extend”, “replace”, “raise”. What to do if the tables we want to write to already exist. Default “extend”. - “if_database_exists”: (str): One of “extend”, “replace”, “raise”. What to do if the database we want to write to already exists. Default “extend”.

  • loglike_kwargs (dict) – Additional keyword arguments for loglike.

  • numdiff_options (dict) – Keyword arguments for the calculation of numerical derivatives for the calculation of standard errors. See Derivatives for details.

  • jacobian (callable or None) – A function that takes params and potentially other keyword arguments and returns the jacobian of loglike[“contributions”] with respect to the params. Note that you only need to pass a Jacobian function if you have a closed form Jacobian. If you pass None, a numerical Jacobian will be calculated.

  • jacobian_kwargs (dict) – Additional keyword arguments for the Jacobian function.

  • hessian (callable or None or False) – A function that takes params and potentially other keyword arguments and returns the Hessian of loglike[“value”] with respect to the params. If you pass None, a numerical Hessian will be calculated. If you pass False, you signal that no Hessian should be calculated. Thus, no result that requires the Hessian will be calculated.

  • hessian_kwargs (dict) – Additional keyword arguments for the Hessian function.

  • design_info (pandas.DataFrame) – DataFrame with one row per observation that contains some or all of the variables “psu” (primary sampling unit), “strata” and “fpc” (finite population corrector). See Robust Likelihood inference for details.

Returns

A LikelihoodResult object.

Return type

LikelihoodResult

estimate_msm
estimagic.estimate_msm(simulate_moments, empirical_moments, moments_cov, params, optimize_options, *, lower_bounds=None, upper_bounds=None, constraints=None, logging=False, log_options=None, simulate_moments_kwargs=None, weights='diagonal', numdiff_options=None, jacobian=None, jacobian_kwargs=None)[source]#

Do a method of simulated moments or indirect inference estimation.

This is a high level interface for our lower level functions for minimization, numerical differentiation, inference and sensitivity analysis. It does the full workflow for MSM or indirect inference estimation with just one function call.

While we have good defaults, you can still configure each aspect of each steps vial the optional arguments of this functions. If you find it easier to do the minimization separately, you can do so and just provide the optimal parameters as params and set optimize_options=False.

Parameters
  • simulate_moments (callable) – Function that takes params and potentially other keyword arguments and returns a pytree with simulated moments. If the function returns a dict containing the key "simulated_moments" we only use the value corresponding to that key. Other entries are stored in the log database if you use logging.

  • empirical_moments (pandas.Series) – A pytree with the same structure as the result of simulate_moments.

  • moments_cov (pandas.DataFrame) – A block-pytree containing the covariance matrix of the empirical moments. This is typically calculated with our get_moments_cov function.

  • params (pytree) – A pytree containing the estimated or start parameters of the model. If the supplied parameters are estimated parameters, set optimize_options to False. Pytrees can be a numpy array, a pandas Series, a DataFrame with “value” column, a float and any kind of (nested) dictionary or list containing these elements. See How to specify params for examples.

  • optimize_options (dict, str or False) – Keyword arguments that govern the numerical optimization. Valid entries are all arguments of minimize() except for those that can be passed explicitly to estimate_msm. If you pass False as optimize_options you signal that params are already the optimal parameters and no numerical optimization is needed. If you pass a str as optimize_options it is used as the algorithm option.

  • lower_bounds (pytree) – A pytree with the same structure as params with lower bounds for the parameters. Can be -np.inf for parameters with no lower bound.

  • upper_bounds (pytree) – As lower_bounds. Can be np.inf for parameters with no upper bound.

  • simulate_moments_kwargs (dict) – Additional keyword arguments for simulate_moments.

  • weights (str) – One of “diagonal” (default), “identity” or “optimal”. Note that “optimal” refers to the asymptotically optimal weighting matrix and is often not a good choice due to large finite sample bias.

  • constraints (list, dict) – List with constraint dictionaries or single dict. See How to specify constraints.

  • logging (pathlib.Path, str or False) – Path to sqlite3 file (which typically has the file extension .db. If the file does not exist, it will be created. The dashboard can only be used when logging is used.

  • log_options (dict) –

    Additional keyword arguments to configure the logging.

    • ”fast_logging” (bool):

      A boolean that determines if “unsafe” settings are used to speed up write processes to the database. This should only be used for very short running criterion functions where the main purpose of the log is a real-time dashboard and it would not be catastrophic to get a corrupted database in case of a sudden system shutdown. If one evaluation of the criterion function (and gradient if applicable) takes more than 100 ms, the logging overhead is negligible.

    • ”if_table_exists” (str):

      One of “extend”, “replace”, “raise”. What to do if the tables we want to write to already exist. Default “extend”.

    • ”if_database_exists” (str):

      One of “extend”, “replace”, “raise”. What to do if the database we want to write to already exists. Default “extend”.

  • numdiff_options (dict) – Keyword arguments for the calculation of numerical derivatives for the calculation of standard errors. See Derivatives for details. Note that by default we increase the step_size by a factor of 2 compared to the rule of thumb for optimal step sizes. This is because many msm criterion functions are slightly noisy.

  • jacobian (callable) – A function that take params and potentially other keyword arguments and returns the jacobian of simulate_moments with respect to the params.

  • jacobian_kwargs (dict) – Additional keyword arguments for the jacobian function.

  • Returns

    dict: The estimated parameters, standard errors and sensitivity measures

    and covariance matrix of the parameters.

get_moments_cov
estimagic.get_moments_cov(data, calculate_moments, *, moment_kwargs=None, bootstrap_kwargs=None)[source]#

Bootstrap the covariance matrix of the moment conditions.

Parameters
  • data (pandas.DataFrame) – DataFrame with empirical data.

  • calculate_moments (callable) – Function that calculates that takes data and moment_kwargs as arguments and returns a 1d numpy array or pandas Series with moment conditions.

  • moment_kwargs (dict) – Additional keyword arguments for calculate_moments.

  • bootstrap_kwargs (dict) – Additional keyword arguments that govern the bootstrapping. Allowed arguments are “n_draws”, “seed”, “n_cores”, “batch_evaluator”, “cluster_by” and “error_handling”. For details see the bootstrap function.

Returns

The covariance matrix of the moment

conditions for msm estimation.

Return type

pandas.DataFrame or numpy.ndarray

lollipop_plot
estimagic.lollipop_plot(data, *, sharex=True, plot_bar=True, n_rows=1, scatterplot_kws=None, barplot_kws=None, combine_plots_in_grid=True, template='simple_white', palette=['rgb(102,194,165)', 'rgb(252,141,98)', 'rgb(141,160,203)', 'rgb(231,138,195)', 'rgb(166,216,84)', 'rgb(255,217,47)', 'rgb(229,196,148)', 'rgb(179,179,179)'])[source]#

Make a lollipop plot.

Parameters
  • data (pandas.DataFrame) – The datapoints to be plotted. The whole data will be

  • need (plotted. Thus if you want to plot just some variables or rows you) –

  • it. (to restrict the dataset before passing) –

  • sharex (bool) – Whether the x-axis is shared across variables, default True.

  • plot_bar (bool) – Whether thin bars are plotted, default True.

  • n_rows (int) – Number of rows for a grid if plots are combined in a grid, default 1. The number of columns is determined automatically.

  • scatterplot_kws (dict) – Keyword arguments to plot the dots of the lollipop plot via the scatter function.

  • barplot_kws (dict) – Keyword arguments to plot the lines of the lollipop plot via the barplot function.

  • combine_plots_in_grid (bool) – decide whether to return a one

  • dictionary (figure containing subplots for each factor pair or a) –

  • True. (of individual plots. Default) –

  • template (str) – The template for the figure. Default is “plotly_white”.

  • palette – The coloring palette for traces. Default is “qualitative.Plotly”.

Returns

The grid plot or dict of individual plots

Return type

plotly.Figure

estimation_table
estimagic.estimation_table(models, *, return_type='dataframe', render_options=None, show_col_names=True, show_col_groups=None, show_index_names=False, show_inference=True, show_stars=True, show_footer=True, custom_param_names=None, custom_col_names=None, custom_col_groups=None, custom_index_names=None, custom_notes=None, confidence_intervals=False, significance_levels=(0.1, 0.05, 0.01), append_notes=True, notes_label='Note:', stats_options=None, number_format=('{0:.3g}', '{0:.5f}', '{0:.4g}'), add_trailing_zeros=True, escape_special_characters=True, siunitx_warning=True)[source]#

Generate html or LaTex tables provided (lists of) of models.

The function can create publication quality tables in various formats from statsmodels or estimagic results.

It allows for extensive customization via optional arguments and almost limitless flexibility when using a two-stage approach where the return_type is set to "render_inputs", the resulting dictionary representation of the table is modified and that modified version is then passed to render_latex or render_html.

The formatting of the numbers in the table is completely configurable via the number_format argument. By default we round to three significant digits (i.e. the three leftmost non-zero digits are displayed). This is very different from other table packages and motivated by the fact that most estimation tables give a wrong feeling of precision by showing too many decimal points.

Parameters
  • models (list) – list of estimation results. The models can come from statmodels or be constructed from the outputs of estimagic.estimate_ml or estimagic.estimate_msm. With a little bit of work it is also possible to construct them out of R or other results. If a model is not a statsmodels results they must be dictionaries with the following entries: “params” (a DataFrame with value column), “info” (a dictionary with summary statistics such as “n_obs”, “rsquared”, …) and “name” (a string), or a DataFrame with value column. If a models is a statsmodels result, model.endog_names is used as name and the rest is extracted from corresponding statsmodels attributes. The model names do not have to be unique but if they are not, models with the same name need to be grouped together.

  • return_type (str) – Can be “dataframe”, “latex”, “html”, “render_inputs” or a file path with the extension .tex or .html. If “render_inputs” is passed, a dictionary with the entries “body”, “footer” and other information is returned. The entries can be modified by the user ( e.g. change formatting, renameof columns or index, …) and then passed to render_latex or render_html. Default “dataframe”.

  • render_options (dict) – a dictionary with keyword arguments that are passed to df.style.to_latex or df.style.to_html, depending on the return_type. The default is None.

  • show_col_names (bool) – If True, the column names are displayed. The default column names are the model names if the model names are unique, otherwise (1), (2), etc.. Default True.

  • show_col_groups (bool) – If True, the column groups are displayed. The default column groups are the model names if the model names are not unique and undefined otherwise. Default None. None means that the column groups are displayed if they are defined.

  • show_index_names (bool) – If True, the index names are displayed. Default False. This is mostly relevant when working with estimagic style params DataFrames with a MultiIndex.

  • show_inference (bool) – If True, inference (standard errors or confidence intervals) are displayed below parameter values. Default True.

  • show_stars (bool) – a boolean variable for displaying significance stars. Default is True.

  • show_footer (bool) – a boolean variable for displaying statistics, e.g. R2, Obs numbers. Default is True. Which statistics are displayed and how they are labeled can be determined via stats_options.

  • custom_param_names (dict) – Dictionary that is used to rename parameters. The keys are the old parameter names or index entries. The values are the new names. Default None.

  • custom_col_names (dict or list) – A list of column names or dict to rename the default column names. The default column names are the model names if the model names are unique, otherwise (1), (2), etc..

  • custom_col_groups (dict or list) – A list of column group or dict to rename the default column groups. The default column groups are the model names if the model names are not unique and undefined otherwise.

  • custom_index_names (dict or list) – Dictionary or list to set the names of the index levels of the parameters. This is mostly relevant when working with estimagic style params DataFrames with a MultiIndex and only used if “index_names” is set to True in the render_options. Default None.

  • custom_notes (list) – A list of strings for additional notes. Default is None.

  • confidence_intervals (bool) – If True, display confidence intervals as inference values. Display standard errors otherwise. Default False.

  • significance_levels (list) – a list of floats for p value’s significance cut-off values. This is used to generate the significance stars. Default is [0.1,0.05,0.01].

  • append_notes (bool) – A boolean variable for printing p value cutoff explanation and additional notes, if applicable. Default is True.

  • notes_label (str) – A sting to print as the title of the notes section, if applicable. Default is ‘Notes’

  • stats_options (dict) – A dictionary that determines which statistics (e.g. R-Squared, No. of Observations) are displayed and how they are labeled. The keys are the names of the statistics inside the model[‘info’] dictionary or attribute names of a statsmodels results object. The values are the new labels to be displayed for those statistics, i.e. the set of the values is used as row names in the table.

  • number_format (int, str, iterable or callable) – A callable, iterable, integer or string that is used to apply string formatter(s) to floats in the table. Default (“{0:.3g}”, “{0:.5f}”, “{0:.4g}”).

  • add_trailing_zeros (bool) – If True, format floats such that they have same number of digits after the decimal point. Default True.

  • siunitx_warning (bool) – If True, print warning about LaTex preamble to add for proper compilation of when working with siunitx package. Default True.

  • escape_special_characters (bool) – If True, replaces special characters in parameter and model names with LaTeX or HTML safe sequences.

Returns

depending on the rerturn type,

data frame with formatted strings, a string for html or latex tables, or a dictionary with statistics and parameters dataframes, and strings for footers is returned. If the return type is a path, the function saves the resulting table at the given path.

Return type

res_table (data frame, str or dictionary)

render_html
estimagic.render_html(body, footer, render_options=None, show_footer=True, append_notes=True, notes_label='Note:', custom_notes=None, significance_levels=(0.1, 0.05, 0.01), show_index_names=False, show_col_names=True, show_col_groups=True, escape_special_characters=True, **kwargs)[source]#

Return estimation table in html format as string.

Parameters
  • body (pandas.DataFrame) – DataFrame with formatted strings of parameter values, inferences (standard errors or confidence intervals, if applicable) and significance stars (if applicable).

  • footer (pandas.DataFrame) – DataFrame with formatted strings of summary statistics (such as number of observations, r-squared, etc.)

  • notes (str) – The html string with notes with additional information (e.g. mapping from pvalues to significance stars) to append to the footer of the estimation table string with LaTex code for the notes section.

  • render_options (dict) – A dictionary with custom kwargs to pass to pd.to_latex(), to update the default options. An example is {header: False} that disables displaying column names.

  • show_footer (bool) – a boolean variable for displaying footer_df. Default True.

  • append_notes (bool) – A boolean variable for printing p value cutoff explanation and additional notes, if applicable. Default is True.

  • notes_label (str) – A sting to print as the title of the notes section, if applicable. Default is ‘Notes’

  • significance_levels (list or tuple) – a list of floats for p value’s significance cutt-off values. Default is [0.1,0.05,0.01].

  • show_index_names (bool) – If True, display index names in the table.

  • show_col_names (bool) – If True, the column names are displayed.

  • show_col_groups (bool) – If True, the column groups are displayed.

  • escape_special_characters (bool) – If True, replace the characters &, <, >, ‘, and ” in parameter and model names with HTML-safe sequences.

Returns

The resulting string with html tabular code.

Return type

html_str (str)

render_latex
estimagic.render_latex(body, footer, render_options=None, show_footer=True, append_notes=True, notes_label='Note:', significance_levels=(0.1, 0.05, 0.01), custom_notes=None, siunitx_warning=True, show_index_names=False, show_col_names=True, show_col_groups=True, escape_special_characters=True)[source]#

Return estimation table in LaTeX format as string.

Parameters
  • body (pandas.DataFrame) – DataFrame with formatted strings of parameter values, inferences (standard errors or confidence intervals, if applicable) and significance stars (if applicable).

  • footer (pandas.DataFrame) – DataFrame with formatted strings of summary statistics (such as number of observations, r-squared, etc.)

  • render_options (dict) – A dictionary with custom kwargs to pass to pd.Styler.to_latex(), to update the default options. An example keyword argument is: - siunitx (bool): If True, the table is structured to be compatible with siunitx package. Default is set to True internally. For the list of all possible arguments, see documentation of pandas.io.formats.style.Styler.to_latex.

  • show_footer (bool) – a boolean variable for displaying footer_df. Default True.

  • append_notes (bool) – A boolean variable for printing p value cutoff explanation and additional notes, if applicable. Default is True.

  • notes_label (str) – A sting to print as the title of the notes section, if applicable. Default is ‘Notes’

  • significance_levels (list or tuple) – a list of floats for p value’s significance cutt-off values. Default is [0.1,0.05,0.01].

  • custom_notes (list) – A list of strings for additional notes. Default is None.

  • siunitx_warning (bool) – If True, print warning about LaTex preamble to add for proper compilation of when working with siunitx package. Default True.

  • show_index_names (bool) – If True, display index names in the table.

  • show_col_names (bool) – If True, the column names are displayed.

  • show_col_groups (bool) – If True, the column groups are displayed.

  • escape_special_characters (bool) – If True, replaces the characters &, %, $, #, _, {, }, ~, ^, and in parameter and model names with LaTeX-safe sequences.

Returns

The resulting string with Latex tabular code.

Return type

latex_str (str)

LikelihoodResult
class estimagic.LikelihoodResult(_params: typing.Any, _internal_estimates: estimagic.parameters.space_conversion.InternalParams, _free_estimates: estimagic.inference.shared.FreeParams, _converter: estimagic.parameters.conversion.Converter, _has_constraints: bool, _optimize_result: typing.Optional[estimagic.optimization.optimize_result.OptimizeResult] = None, _jacobian: typing.Optional[typing.Any] = None, _no_jacobian_reason: typing.Optional[str] = None, _hessian: typing.Optional[typing.Any] = None, _no_hessian_reason: typing.Optional[str] = None, _internal_jacobian: typing.Optional[numpy.ndarray] = None, _internal_hessian: typing.Optional[numpy.ndarray] = None, _design_info: typing.Optional[pandas.core.frame.DataFrame] = None, _cache: typing.Dict = <factory>)[source]#

Likelihood estimation results object.

se(method='jacobian', n_samples=10000, bounds_handling='clip', seed=None)[source]#

Calculate standard errors.

Parameters
  • method (str) – One of “jacobian”, “hessian”, “robust”, “cluster_robust”, “strata_robust”. Default “jacobian”. “cluster_robust” is only available if design_info containts a columns called “psu” that identifies the primary sampling unit. “strata_robust” is only available if the columns “strata”, “fpc” and “psu” are in design_info.

  • n_samples (int) – Number of samples used to transform the covariance matrix of the internal parameter vector into the covariance matrix of the external parameters. For background information about internal and external params see How constraints are implemented. This is only used if you are using constraints.

  • bounds_handling (str) – One of “clip”, “raise”, “ignore”. Determines how bounds are handled. If “clip”, confidence intervals are clipped at the bounds. Standard errors are only adjusted if a sampling step is necessary due to additional constraints. If “raise” and any lower or upper bound is binding, we raise an Error. If “ignore”, boundary problems are simply ignored.

  • seed (int) – Seed for the random number generator. Only used if there are transforming constraints.

Returns

A pytree with the same structure as params containing standard errors

for the parameter estimates.

Return type

Any

cov(method='jacobian', n_samples=10000, bounds_handling='clip', return_type='pytree', seed=None)[source]#

Calculate the variance-covariance (matrix) of the estimated parameters.

Parameters
  • method (str) – One of “jacobian”, “hessian”, “robust”, “cluster_robust”, “strata_robust”. Default “jacobian”. “cluster_robust” is only available if design_info containts a columns called “psu” that identifies the primary sampling unit. “strata_robust” is only available if the columns “strata”, “fpc” and “psu” are in design_info.

  • n_samples (int) – Number of samples used to transform the covariance matrix of the internal parameter vector into the covariance matrix of the external parameters. For background information about internal and external params see How constraints are implemented. This is only used if you are using constraints.

  • bounds_handling (str) – One of “clip”, “raise”, “ignore”. Determines how bounds are handled. If “clip”, confidence intervals are clipped at the bounds. Standard errors are only adjusted if a sampling step is necessary due to additional constraints. If “raise” and any lower or upper bound is binding, we raise an Error. If “ignore”, boundary problems are simply ignored.

  • return_type (str) – One of “pytree”, “array” or “dataframe”. Default pytree. If “array”, a 2d numpy array with the covariance is returned. If “dataframe”, a pandas DataFrame with parameter names in the index and columns are returned.

  • seed (int) – Seed for the random number generator. Only used if there are transforming constraints.

Returns

The covariance matrix of the estimated parameters as block-pytree,

numpy.ndarray or pandas.DataFrame.

Return type

Any

summary(method='jacobian', n_samples=10000, ci_level=0.95, bounds_handling='clip', seed=None)[source]#

Create a summary of estimation results.

Parameters
  • method (str) – One of “jacobian”, “hessian”, “robust”, “cluster_robust”, “strata_robust”. Default “jacobian”. “cluster_robust” is only available if design_info containts a columns called “psu” that identifies the primary sampling unit. “strata_robust” is only available if the columns “strata”, “fpc” and “psu” are in design_info.

  • ci_level (float) – Confidence level for the calculation of confidence intervals. The default is 0.95.

  • n_samples (int) – Number of samples used to transform the covariance matrix of the internal parameter vector into the covariance matrix of the external parameters. For background information about internal and external params see How constraints are implemented. This is only used if you are using constraints.

  • bounds_handling (str) – One of “clip”, “raise”, “ignore”. Determines how bounds are handled. If “clip”, confidence intervals are clipped at the bounds. Standard errors are only adjusted if a sampling step is necessary due to additional constraints. If “raise” and any lower or upper bound is binding, we raise an Error. If “ignore”, boundary problems are simply ignored.

  • seed (int) – Seed for the random number generator. Only used if there are transforming constraints.

Returns

The estimation summary as pytree of DataFrames.

Return type

Any

ci(method='jacobian', n_samples=10000, ci_level=0.95, bounds_handling='clip', seed=None)[source]#

Calculate confidence intervals.

Parameters
  • method (str) – One of “jacobian”, “hessian”, “robust”, “cluster_robust”, “strata_robust”. Default “jacobian”. “cluster_robust” is only available if design_info containts a columns called “psu” that identifies the primary sampling unit. “strata_robust” is only available if the columns “strata”, “fpc” and “psu” are in design_info.

  • ci_level (float) – Confidence level for the calculation of confidence intervals. The default is 0.95.

  • n_samples (int) – Number of samples used to transform the covariance matrix of the internal parameter vector into the covariance matrix of the external parameters. For background information about internal and external params see How constraints are implemented. This is only used if you are using constraints.

  • bounds_handling (str) – One of “clip”, “raise”, “ignore”. Determines how bounds are handled. If “clip”, confidence intervals are clipped at the bounds. Standard errors are only adjusted if a sampling step is necessary due to additional constraints. If “raise” and any lower or upper bound is binding, we raise an Error. If “ignore”, boundary problems are simply ignored.

  • seed (int) – Seed for the random number generator. Only used if there are transforming constraints.

Returns

Pytree with the same structure as params containing lower bounds of

confidence intervals.

Any: Pytree with the same structure as params containing upper bounds of

confidence intervals.

Return type

Any

p_values(method='jacobian', n_samples=10000, bounds_handling='clip', seed=None)[source]#

Calculate p-values.

Parameters
  • method (str) – One of “jacobian”, “hessian”, “robust”, “cluster_robust”, “strata_robust”. Default “jacobian”. “cluster_robust” is only available if design_info containts a columns called “psu” that identifies the primary sampling unit. “strata_robust” is only available if the columns “strata”, “fpc” and “psu” are in design_info.

  • ci_level (float) – Confidence level for the calculation of confidence intervals. The default is 0.95.

  • n_samples (int) – Number of samples used to transform the covariance matrix of the internal parameter vector into the covariance matrix of the external parameters. For background information about internal and external params see How constraints are implemented. This is only used if you are using constraints.

  • bounds_handling (str) – One of “clip”, “raise”, “ignore”. Determines how bounds are handled. If “clip”, confidence intervals are clipped at the bounds. Standard errors are only adjusted if a sampling step is necessary due to additional constraints. If “raise” and any lower or upper bound is binding, we raise an Error. If “ignore”, boundary problems are simply ignored.

  • seed (int) – Seed for the random number generator. Only used if there are transforming constraints.

Returns

Pytree with the same structure as params containing p-values. Any: Pytree with the same structure as params containing p-values.

Return type

Any

to_pickle(path)[source]#

Save the LikelihoodResult object to pickle.

Parameters

path (str, pathlib.Path) – A str or pathlib.path ending in .pkl or .pickle.

MomentsResult
class estimagic.MomentsResult(_params: typing.Any, _internal_estimates: estimagic.parameters.space_conversion.InternalParams, _free_estimates: estimagic.inference.shared.FreeParams, _weights: typing.Any, _converter: estimagic.parameters.conversion.Converter, _internal_moments_cov: numpy.ndarray, _internal_weights: numpy.ndarray, _internal_jacobian: numpy.ndarray, _empirical_moments: typing.Any, _has_constraints: bool, _optimize_result: typing.Optional[estimagic.optimization.optimize_result.OptimizeResult] = None, _jacobian: typing.Optional[typing.Any] = None, _no_jacobian_reason: typing.Optional[str] = None, _cache: typing.Dict = <factory>)[source]#

Method of moments estimation results object.

se(method='robust', n_samples=10000, bounds_handling='clip', seed=None)[source]#

Calculate standard errors.

Parameters
  • method (str) – One of “robust”, “optimal”. Despite the name, “optimal” is not recommended in finite samples and “optimal” standard errors are only valid if the asymptotically optimal weighting matrix has been used. It is only supported because it is needed to calculate sensitivity measures.

  • n_samples (int) – Number of samples used to transform the covariance matrix of the internal parameter vector into the covariance matrix of the external parameters. For background information about internal and external params see How constraints are implemented. This is only used if you are using constraints.

  • bounds_handling (str) – One of “clip”, “raise”, “ignore”. Determines how bounds are handled. If “clip”, confidence intervals are clipped at the bounds. Standard errors are only adjusted if a sampling step is necessary due to additional constraints. If “raise” and any lower or upper bound is binding, we raise an Error. If “ignore”, boundary problems are simply ignored.

  • seed (int) – Seed for the random number generator. Only used if there are transforming constraints.

Returns

A pytree with the same structure as params containing standard errors

for the parameter estimates.

Return type

Any

cov(method='robust', n_samples=10000, bounds_handling='clip', return_type='pytree', seed=None)[source]#

Calculate the variance-covariance matrix of the estimated parameters.

Parameters
  • method (str) – One of “robust”, “optimal”. Despite the name, “optimal” is not recommended in finite samples and “optimal” standard errors are only valid if the asymptotically optimal weighting matrix has been used. It is only supported because it is needed to calculate sensitivity measures.

  • n_samples (int) – Number of samples used to transform the covariance matrix of the internal parameter vector into the covariance matrix of the external parameters. For background information about internal and external params see How constraints are implemented. This is only used if you are using constraints.

  • bounds_handling (str) – One of “clip”, “raise”, “ignore”. Determines how bounds are handled. If “clip”, confidence intervals are clipped at the bounds. Standard errors are only adjusted if a sampling step is necessary due to additional constraints. If “raise” and any lower or upper bound is binding, we raise an Error. If “ignore”, boundary problems are simply ignored.

  • return_type (str) – One of “pytree”, “array” or “dataframe”. Default pytree. If “array”, a 2d numpy array with the covariance is returned. If “dataframe”, a pandas DataFrame with parameter names in the index and columns are returned.

  • seed (int) – Seed for the random number generator. Only used if there are transforming constraints.

Returns

The covariance matrix of the estimated parameters as block-pytree or

numpy array.

Return type

Any

summary(method='robust', n_samples=10000, ci_level=0.95, bounds_handling='clip', seed=None)[source]#

Create a summary of estimation results.

Parameters
  • method (str) – One of “robust”, “optimal”. Despite the name, “optimal” is not recommended in finite samples and “optimal” standard errors are only valid if the asymptotically optimal weighting matrix has been used. It is only supported because it is needed to calculate sensitivity measures.

  • ci_level (float) – Confidence level for the calculation of confidence intervals. The default is 0.95.

  • n_samples (int) – Number of samples used to transform the covariance matrix of the internal parameter vector into the covariance matrix of the external parameters. For background information about internal and external params see How constraints are implemented. This is only used if you are using constraints.

  • bounds_handling (str) – One of “clip”, “raise”, “ignore”. Determines how bounds are handled. If “clip”, confidence intervals are clipped at the bounds. Standard errors are only adjusted if a sampling step is necessary due to additional constraints. If “raise” and any lower or upper bound is binding, we raise an Error. If “ignore”, boundary problems are simply ignored.

  • seed (int) – Seed for the random number generator. Only used if there are transforming constraints.

Returns

The estimation summary as pytree of DataFrames.

Return type

Any

ci(method='robust', n_samples=10000, ci_level=0.95, bounds_handling='clip', seed=None)[source]#

Calculate confidence intervals.

Parameters
  • method (str) – One of “robust”, “optimal”. Despite the name, “optimal” is not recommended in finite samples and “optimal” standard errors are only valid if the asymptotically optimal weighting matrix has been used. It is only supported because it is needed to calculate sensitivity measures.

  • ci_level (float) – Confidence level for the calculation of confidence intervals. The default is 0.95.

  • n_samples (int) – Number of samples used to transform the covariance matrix of the internal parameter vector into the covariance matrix of the external parameters. For background information about internal and external params see How constraints are implemented. This is only used if you are using constraints.

  • bounds_handling (str) – One of “clip”, “raise”, “ignore”. Determines how bounds are handled. If “clip”, confidence intervals are clipped at the bounds. Standard errors are only adjusted if a sampling step is necessary due to additional constraints. If “raise” and any lower or upper bound is binding, we raise an Error. If “ignore”, boundary problems are simply ignored.

  • seed (int) – Seed for the random number generator. Only used if there are transforming constraints.

Returns

Pytree with the same structure as params containing lower bounds of

confidence intervals.

Any: Pytree with the same structure as params containing upper bounds of

confidence intervals.

Return type

Any

p_values(method='robust', n_samples=10000, bounds_handling='clip', seed=None)[source]#

Calculate p-values.

Parameters
  • method (str) – One of “robust”, “optimal”. Despite the name, “optimal” is not recommended in finite samples and “optimal” standard errors are only valid if the asymptotically optimal weighting matrix has been used. It is only supported because it is needed to calculate sensitivity measures.

  • n_samples (int) – Number of samples used to transform the covariance matrix of the internal parameter vector into the covariance matrix of the external parameters. For background information about internal and external params see How constraints are implemented. This is only used if you are using constraints.

  • bounds_handling (str) – One of “clip”, “raise”, “ignore”. Determines how bounds are handled. If “clip”, confidence intervals are clipped at the bounds. Standard errors are only adjusted if a sampling step is necessary due to additional constraints. If “raise” and any lower or upper bound is binding, we raise an Error. If “ignore”, boundary problems are simply ignored.

  • seed (int) – Seed for the random number generator. Only used if there are transforming constraints.

Returns

Pytree with the same structure as params containing p-values. Any: Pytree with the same structure as params containing p-values.

Return type

Any

sensitivity(kind='bias', n_samples=10000, bounds_handling='clip', seed=None, return_type='pytree')[source]#

Calculate sensitivity measures for moments estimates.

The sensitivity measures are based on the following papers:

Andrews, Gentzkow & Shapiro (2017, Quarterly Journal of Economics)

Honore, Jorgensen & de Paula (https://onlinelibrary.wiley.com/doi/full/10.1002/jae.2779)

In the papers the different kinds of sensitivity measures are just called m1, e2, e3, e4, e5 and e6. We try to give them more informative names, but list the original names for references.

Parameters
  • kind (str) –

    The following kinds are supported:

    • ”bias”:

      Origally m1. How strongly would the parameter estimates be biased if the kth moment was misspecified, i.e not zero in expectation?

    • ”noise_fundamental”:

      Originally e2. How much precision would be lost if the kth moment was subject to a little additional noise if the optimal weighting matrix was used?

    • ”noise”:

      Originally e3. How much precision would be lost if the kth moment was subjet to a little additional noise?

    • ”removal”:

      Originally e4. How much precision would be lost if the kth moment was excluded from the estimation?

    • ”removal_fundamental”:

      Originally e5. How much precision would be lost if the kth moment was excluded from the estimation if the asymptotically optimal weighting matrix was used.

    • ”weighting”:

      Originally e6. How would the precision change if the weight of the kth moment is increased a little?

  • n_samples (int) – Number of samples used to transform the covariance matrix of the internal parameter vector into the covariance matrix of the external parameters. For background information about internal and external params see How constraints are implemented. This is only used if you are using constraints.

  • bounds_handling (str) – One of “clip”, “raise”, “ignore”. Determines how bounds are handled. If “clip”, confidence intervals are clipped at the bounds. Standard errors are only adjusted if a sampling step is necessary due to additional constraints. If “raise” and any lower or upper bound is binding, we raise an Error. If “ignore”, boundary problems are simply ignored.

  • seed (int) – Seed for the random number generator. Only used if there are transforming constraints.

  • return_type (str) – One of “array”, “dataframe” or “pytree”. Default pytree. If your params or moments have a very nested format, return_type “dataframe” might be the better choice.

Returns

The sensitivity measure as a pytree, numpy array or DataFrame.

In 2d formats, the sensitivity measures have one row per estimated parameter and one column per moment.

Return type

Any

to_pickle(path)[source]#

Save the MomentsResult object to pickle.

Parameters

path (str, pathlib.Path) – A str or pathlib.path ending in .pkl or .pickle.

Bootstrap#

bootstrap
estimagic.bootstrap(outcome, data, *, existing_result=None, outcome_kwargs=None, n_draws=1000, cluster_by=None, seed=None, n_cores=1, error_handling='continue', batch_evaluator=<function joblib_batch_evaluator>)[source]#

Use the bootstrap to calculate inference quantities.

Parameters
  • outcome (callable) – A function that computes the statistic of interest.

  • data (pd.DataFrame) – Dataset.

  • existing_result (BootstrapResult) – An existing BootstrapResult object from a previous call of bootstrap(). Default is None.

  • outcome_kwargs (dict) – Additional keyword arguments for outcome.

  • n_draws (int) – Number of bootstrap samples to draw. If len(existing_outcomes) >= n_draws, a random subset of existing_outcomes is used.

  • cluster_by (str) – Column name of variable to cluster by or None.

  • seed (Union[None, int, numpy.random.Generator]) – If seed is None or int the numpy.random.default_rng is used seeded with seed. If seed is already a Generator instance then that instance is used.

  • n_cores (int) – number of jobs for parallelization.

  • error_handling (str) – One of “continue”, “raise”. Default “continue” which means that bootstrap estimates are only calculated for those samples where no errors occur and a warning is produced if any error occurs.

  • batch_evaluator (str or Callable) – Name of a pre-implemented batch evaluator (currently ‘joblib’ and ‘pathos_mp’) or Callable with the same interface as the estimagic batch_evaluators. See Batch evaluators.

Returns

A BootstrapResult object storing information on summary

statistics, the covariance matrix, and estimated boostrap outcomes.

Return type

BootstrapResult

BootstrapResult
class estimagic.BootstrapResult(_base_outcome: Any, _internal_outcomes: numpy.ndarray, _internal_cov: numpy.ndarray)[source]#
property base_outcome#

Returns the base outcome statistic(s).

Returns

Pytree of base outcomes, i.e. the outcome statistic(s) evaluated

on the original data set.

Return type

pytree

property outcomes#

Returns the estimated bootstrap outcomes.

Returns

The boostrap outcomes as a list of pytrees.

Return type

List[Any]

se()[source]#

Calculate standard errors.

Returns

The standard errors of the estimated parameters as a block-pytree,

numpy.ndarray, or pandas.DataFrame.

Return type

Any

cov(return_type='pytree')[source]#

Calculate the variance-covariance matrix of the estimated parameters.

Parameters

return_type (str) – One of “pytree”, “array” or “dataframe”. Default pytree. If “array”, a 2d numpy array with the covariance is returned. If “dataframe”, a pandas DataFrame with parameter names in the index and columns are returned. The default is “pytree”.

Returns

The covariance matrix of the estimated parameters as a block-pytree,

numpy.ndarray, or pandas.DataFrame.

Return type

Any

ci(ci_method='percentile', ci_level=0.95)[source]#

Calculate confidence intervals.

Parameters
  • ci_method (str) – Method of choice for computing confidence intervals. The default is “percentile”.

  • ci_level (float) – Confidence level for the calculation of confidence intervals. The default is 0.95.

Returns

Pytree with the same structure as base_outcome containing lower

bounds of confidence intervals.

Any: Pytree with the same structure as base_outcome containing upper

bounds of confidence intervals.

Return type

Any

p_values()[source]#

Calculate p-values.

Returns

A pytree with the same structure as base_outcome containing p-values

for the parameter estimates.

Return type

Any

summary(ci_method='percentile', ci_level=0.95)[source]#

Create a summary of bootstrap results.

Parameters
  • ci_method (str) – Method of choice for confidence interval computation. The default is “percentile”.

  • ci_level (float) – Confidence level for the calculation of confidence intervals. The default is 0.95.

Returns

The estimation summary as a DataFrame containing information

on the mean, standard errors, as well as the confidence intervals. Soon this will be a pytree.

Return type

pd.DataFrame

Benchmarks#

get_benchmark_problems
estimagic.get_benchmark_problems(name, *, additive_noise=False, additive_noise_options=None, multiplicative_noise=False, multiplicative_noise_options=None, scaling=False, scaling_options=None, seed=None, exclude=None)[source]#

Get a dictionary of test problems for a benchmark.

Parameters
  • name (str) – The name of the set of test problems. Currently “more_wild” is the only supported one.

  • additive_noise (bool) – Whether to add additive noise to the problem. Default False.

  • additive_noise_options (dict or None) – Specifies the amount and distribution of the addititve noise added to the problem. Has the entries: - distribition (str): One of “normal”, “gumbel”, “uniform”, “logistic”, “laplace”. Default “normal”. - std (float): The standard deviation of the noise. This works for all distributions, even if those distributions are normally not specified via a standard deviation (e.g. uniform). - correlation (float): Number between 0 and 1 that specifies the auto correlation of the noise.

  • multiplicative_noise (bool) – Whether to add multiplicative noise to the problem. Default False.

  • multiplicative_noise_options (dict or None) – Specifies the amount and distribition of the multiplicative noise added to the problem. Has entries: - distribition (str): One of “normal”, “gumbel”, “uniform”, “logistic”, “laplace”. Default “normal”. - std (float): The standard deviation of the noise. This works for all distributions, even if those distributions are normally not specified via a standard deviation (e.g. uniform). - correlation (float): Number between 0 and 1 that specifies the auto correlation of the noise. - clipping_value (float): A non-negative float. Multiplicative noise becomes zero if the function value is zero. To avoid this, we do not implement multiplicative noise as f_noisy = f * epsilon but by f_noisy = f + (epsilon - 1) * f_clipped` where f_clipped is bounded away from zero from both sides by the clipping value.

  • scaling (bool) – Whether the parameter space of the problem should be rescaled.

  • scaling_options (dict) – Dict containing the keys “min_scale”, and “max_scale”. If scaling is True, the parameters the optimizer sees are the standard parameters multiplied by np.linspace(min_scale, max_scale, len(params)). If min_scale and max_scale have very different orders of magnitude, the problem becomes harder to solve for many optimizers.

  • seed (Union[None, int, numpy.random.Generator]) – If seed is None or int the numpy.random.default_rng is used seeded with seed. If seed is already a Generator instance then that instance is used.

  • exclude (str or List) – Problems to exclude.

Returns

Nested dictionary with benchmark problems of the structure:

{“name”: {“inputs”: {…}, “solution”: {…}, “info”: {…}}} where “inputs” are keyword arguments for minimize such as the criterion function and start parameters. “solution” contains the entries “params” and “value” and “info” might contain information about the test problem.

Return type

dict

run_benchmark
estimagic.run_benchmark(problems, optimize_options, *, batch_evaluator='joblib', n_cores=1, error_handling='continue', max_criterion_evaluations=1000, disable_convergence=True)[source]#

Run problems with different optimize options.

Parameters
  • problems (dict) – Nested dictionary with benchmark problems of the structure: {“name”: {“inputs”: {…}, “solution”: {…}, “info”: {…}}} where “inputs” are keyword arguments for minimize such as the criterion function and start parameters. “solution” contains the entries “params” and “value” and “info” might contain information about the test problem.

  • optimize_options (list or dict) – Either a list of algorithms or a Nested dictionary that maps a name for optimizer settings (e.g. "lbfgsb_strict_criterion") to a dictionary of keyword arguments for arguments for minimize (e.g. {"algorithm": "scipy_lbfgsb", "algo_options": {"convergence.relative_criterion_tolerance": 1e-12}}). Alternatively, the values can just be an algorithm which is then benchmarked at default settings.

  • batch_evaluator (str or callable) – See Batch evaluators.

  • n_cores (int) – Number of optimizations that is run in parallel. Note that in addition to that an optimizer might parallelize.

  • error_handling (str) – One of “raise”, “continue”.

  • max_criterion_evaluations (int) – Shortcut to set the maximum number of criterion evaluations instead of passing them in via algo options. In case an optimizer does not support this stopping criterion, we also use this as max iterations.

  • disable_convergence (bool) – If True, we set extremely strict convergence convergence criteria by default, such that most optimizers will exploit their full computation budget set by max_criterion_evaluations.

Returns

Nested Dictionary with information on the benchmark run. The outer keys

are tuples where the first entry is the name of the problem and the second the name of the optimize options. The values are dicts with the entries: “params_history”, “criterion_history”, “time_history” and “solution”.

Return type

dict

profile_plot
estimagic.profile_plot(problems, results, *, runtime_measure='n_evaluations', normalize_runtime=False, stopping_criterion='y', x_precision=0.0001, y_precision=0.0001, template='simple_white')[source]#

Compare optimizers over a problem set.

This plot answers the question: What percentage of problems can each algorithm solve within a certain runtime budget?

The runtime budget is plotted on the x axis and the share of problems each algorithm solved on the y axis.

Thus, algorithms that are very specialized and perform well on some share of problems but are not able to solve more problems with a larger computational budget will have steep increases and then flat lines. Algorithms that are robust but slow, will have low shares in the beginning but reach very high.

Note that failing to converge according to the given stopping_criterion and precisions is scored as needing an infinite computational budget.

For details, see the description of performance and data profiles by Moré and Wild (2009).

Parameters
  • problems (dict) – estimagic benchmarking problems dictionary. Keys are the problem names. Values contain information on the problem, including the solution value.

  • results (dict) – estimagic benchmarking results dictionary. Keys are tuples of the form (problem, algorithm), values are dictionaries of the collected information on the benchmark run, including ‘criterion_history’ and ‘time_history’.

  • runtime_measure (str) – “n_evaluations”, “n_batches” or “walltime”. This is the runtime until the desired convergence was reached by an algorithm. This is called performance measure by Moré and Wild (2009).

  • normalize_runtime (bool) – If True the runtime each algorithm needed for each problem is scaled by the time the fastest algorithm needed. If True, the resulting plot is what Moré and Wild (2009) called data profiles.

  • stopping_criterion (str) – one of “x_and_y”, “x_or_y”, “x”, “y”. Determines how convergence is determined from the two precisions.

  • x_precision (float or None) – how close an algorithm must have gotten to the true parameter values (as percent of the Euclidean distance between start and solution parameters) before the criterion for clipping and convergence is fulfilled.

  • y_precision (float or None) – how close an algorithm must have gotten to the true criterion values (as percent of the distance between start and solution criterion value) before the criterion for clipping and convergence is fulfilled.

  • template (str) – The template for the figure. Default is “plotly_white”.

Returns

plotly.Figure

convergence_plot
estimagic.convergence_plot(problems, results, *, problem_subset=None, algorithm_subset=None, n_cols=2, distance_measure='criterion', monotone=True, normalize_distance=True, runtime_measure='n_evaluations', stopping_criterion='y', x_precision=0.0001, y_precision=0.0001, combine_plots_in_grid=True, template='simple_white', palette=['#636EFA', '#EF553B', '#00CC96', '#AB63FA', '#FFA15A', '#19D3F3', '#FF6692', '#B6E880', '#FF97FF', '#FECB52'])[source]#

Plot convergence of optimizers for a set of problems.

This creates a grid of plots, showing the convergence of the different algorithms on each problem. The faster a line falls, the faster the algorithm improved on the problem. The algorithm converged where its line reaches 0 (if normalize_distance is True) or the horizontal blue line labeled “true solution”.

Each plot shows on the x axis the runtime_measure, which can be walltime, number of evaluations or number of batches. Each algorithm’s convergence is a line in the plot. Convergence can be measured by the criterion value of the particular time/evaluation. The convergence can be made monotone (i.e. always taking the bast value so far) or normalized such that the distance from the start to the true solution is one.

Parameters
  • problems (dict) – estimagic benchmarking problems dictionary. Keys are the problem names. Values contain information on the problem, including the solution value.

  • results (dict) – estimagic benchmarking results dictionary. Keys are tuples of the form (problem, algorithm), values are dictionaries of the collected information on the benchmark run, including ‘criterion_history’ and ‘time_history’.

  • problem_subset (list, optional) – List of problem names. These must be a subset of the keys of the problems dictionary. If provided the convergence plot is only created for the problems specified in this list.

  • algorithm_subset (list, optional) – List of algorithm names. These must be a subset of the keys of the optimizer_options passed to run_benchmark. If provided only the convergence of the given algorithms are shown.

  • n_cols (int) – number of columns in the plot of grids. The number of rows is determined automatically.

  • distance_measure (str) – One of “criterion”, “parameter_distance”.

  • monotone (bool) – If True the best found criterion value so far is plotted. If False the particular criterion evaluation of that time is used.

  • normalize_distance (bool) – If True the progress is scaled by the total distance between the start value and the optimal value, i.e. 1 means the algorithm is as far from the solution as the start value and 0 means the algorithm has reached the solution value.

  • runtime_measure (str) – “n_evaluations”, “walltime” or “n_batches”.

  • stopping_criterion (str) – “x_and_y”, “x_or_y”, “x”, “y” or None. If None, no clipping is done.

  • x_precision (float or None) – how close an algorithm must have gotten to the true parameter values (as percent of the Euclidean distance between start and solution parameters) before the criterion for clipping and convergence is fulfilled.

  • y_precision (float or None) – how close an algorithm must have gotten to the true criterion values (as percent of the distance between start and solution criterion value) before the criterion for clipping and convergence is fulfilled.

  • combine_plots_in_grid (bool) – decide whether to return a one figure containing subplots for each factor pair or a dictionary of individual plots. Default True.

  • template (str) – The template for the figure. Default is “plotly_white”.

  • palette – The coloring palette for traces. Default is “qualitative.Plotly”.

Returns

The grid plot or dict of individual plots

Return type

plotly.Figure

Log reading#

OptimizeLogReader
class estimagic.OptimizeLogReader(path: Union[str, pathlib.Path])[source]#

Read information about an optimization from a sqlite database.

Other:#