mrftools.opt module

Optimization utility class containing various optimizers and utility objects for callback functions

class mrftools.opt.ObjectivePlotter(func, grad=None)

Bases: object

Class to generate a plot of the objective function during the callback

callback(x)

Plot the current objectvie value and the current solution, and prints diagnostic information about the current solution, objective, and gradient, when available. :param x: current iterate :return:

class mrftools.opt.WeightRecord

Bases: object

Class used to store solutions during optimization. Used to generate a callback function that will store the solution passed in. Useful for diagnostics, but in production, usually suboptimal solutions don’t need to be saved.

callback(x)

Save x into the WeightRecord with a timestamp

Parameters:x – vector to be saved into the weight record Returns:

mrftools.opt.ada_grad(func, grad, x, args={}, callback=None)

Adagrad adaptive gradient optimizer

Parameters:

• func – function to be minimized (used here only to update the gradient)
• grad – gradient function that returns the gradient of the function to be minimized
• x – vector initial value of value being optimized over
• args – arguments with optimizer options and for the func and grad functions
• callback – function to be called with the current iterate each iteration

Returns:

optimized solution

mrftools.opt.adam(func, grad, x, args={}, callback=None)

Adam adaptive gradient optimizer :param func: function to be minimized (used here only to update the gradient) :param grad: gradient function that returns the gradient of the function to be minimized :param x: vector initial value of value being optimized over :param args: arguments with optimizer options and for the func and grad functions :param callback: function to be called with the current iterate each iteration :return: optimized solution

mrftools.opt.lbfgs(func, grad, x, args={}, callback=None)

Adapter for scipy’s standard minimize function, which defaults to using the LBFGS-B optimizer

Parameters:

• func – function to be minimized (used here only to update the gradient)
• grad – gradient function that returns the gradient of the function to be minimized
• x – vector initial value of value being optimized over
• args – arguments with optimizer options and for the func and grad functions
• callback – function to be called with the current iterate each iteration

Returns:

optimized solution

mrftools.opt.rms_prop(func, grad, x, args={}, callback=None)

RMSProp adaptive gradient optimizer

Parameters:

• func – function to be minimized (used here only to update the gradient)
• grad – gradient function that returns the gradient of the function to be minimized
• x – vector initial value of value being optimized over
• args – arguments with optimizer options and for the func and grad functions
• callback – function to be called with the current iterate each iteration

Returns:

optimized solution

mrftools.opt.sgd(func, grad, x, args={}, callback=None)

Stochastic gradient descent with a linear rate decay :param func: function to be minimized (used here only to update the gradient) :param grad: gradient function that returns the gradient of the function to be minimized :param x: vector initial value of value being optimized over :param args: arguments with optimizer options and for the func and grad functions :param callback: function to be called with the current iterate each iteration :return: optimized solution