How to use logging#

optimagic can keep a persistent log of the parameter and criterion values tried out by an optimizer in a sqlite database.

Turn logging on or off#

To enable logging, it suffices to provide a path to an sqlite database when calling maximize or minimize. The database does not have to exist, optimagic will generate it for you.

from pathlib import Path

import numpy as np

import optimagic as om
def sphere(params):
    return params @ params
# Remove the log file if it exists (just needed for the example)
log_file = Path("my_log.db")
if log_file.exists():
    log_file.unlink()

res = om.minimize(
    fun=sphere,
    params=np.arange(5),
    algorithm="scipy_lbfgsb",
    logging="my_log.db",
)

In case the SQLite file already exists, this will raise a FileExistsError to prevent from accidentally polluting an existing database. If you want to reuse an existing database on purpose, you must explicitly provide the corresponding option for if_database_exists:

log_options = om.SQLiteLogOptions(
    "my_log.db", if_database_exists=om.ExistenceStrategy.EXTEND
)

res = om.minimize(
    fun=sphere,
    params=np.arange(5),
    algorithm="scipy_lbfgsb",
    logging=log_options,
)

Make logging faster#

By default, we use a very safe mode of sqlite that makes it almost impossible to corrupt the database. Even if your computer is suddenly shut down or unplugged.

However, this makes writing logs rather slow, which becomes notable when the criterion function is very fast.

In that case, you can enable fast_logging, which is still quite safe!

log_options = om.SQLiteLogOptions(
    "my_log.db",
    fast_logging=True,
    if_database_exists=om.ExistenceStrategy.REPLACE,
)

res = om.minimize(
    fun=sphere,
    params=np.arange(5),
    algorithm="scipy_lbfgsb",
    logging=log_options,
)

Reading the log#

To read the log after an optimization, extract the logger from the optimization result:

reader = res.logger

Alternatively, you can create the reader like this:

reader = om.SQLiteLogReader("my_log.db")

Read the start params

reader.read_start_params()
array([0, 1, 2, 3, 4])

Read a specific iteration (use -1 for the last)

reader.read_iteration(-1)
IterationStateWithId(params=array([ 0.00000000e+00, -2.19792136e-07, -4.01986529e-08, -1.26862247e-07,
       -2.06263028e-07]), timestamp=18396.33763913, scalar_fun=1.08562981500731e-13, valid=True, raw_fun={'value': np.float64(1.08562981500731e-13), 'info': None}, step=1, exceptions=None, rowid=3)

Read the full history

reader.read_history().keys()
dict_keys(['params', 'fun', 'time'])

Plot the history from a log#

fig = om.criterion_plot("my_log.db")
fig.show(renderer="png")
../_images/6e146a226c93cc688a0f7e1af82243f57b9ee080527fe303cf03dd4f494f379e.png
fig = om.params_plot("my_log.db", selector=lambda x: x[1:3])
fig.show(renderer="png")
../_images/19138b6dc204cd53cb13c258257520740dba10f65c04e0f660b17f956043849d.png