copt.minimize_saga

copt.minimize_saga(f_deriv, A, b, x0, step_size, prox=None, alpha=0, max_iter=500, tol=1e-06, verbose=1, callback=None)

Stochastic average gradient augmented (SAGA) algorithm.

This algorithm can solve linearly-parametrized loss functions of the form

minimize_x sum_{i}^n_samples f(A_i^T x, b_i) + alpha ||x||_2^2 + g(x)

where g is a function for which we have access to its proximal operator.

Warning

This function is experimental, API is likely to change.

Parameters

• f – loss functions.

• x0 – np.ndarray or None, optional Starting point for optimization.

• step_size – float or None, optional Step size for the optimization. If None is given, this will be estimated from the function f.

• max_iter – int Maximum number of passes through the data in the optimization.

• tol – float Tolerance criterion. The algorithm will stop whenever the norm of the gradient mapping (generalization of the gradient for nonsmooth optimization) is below tol.

• verbose – bool Verbosity level. True might print some messages.

• trace – bool Whether to trace convergence of the function, useful for plotting and/or debugging. If ye, the result will have extra members trace_func, trace_time.

Returns

OptimizeResult

The optimization result represented as a scipy.optimize.OptimizeResult object. Important attributes are: x the solution array, success a Boolean flag indicating if the optimizer exited successfully and message which describes the cause of the termination. See scipy.optimize.OptimizeResult for a description of other attributes.

Return type

opt

References

This variant of the SAGA algorithm is described in:

“Breaking the Nonsmooth Barrier: A Scalable Parallel Method for Composite Optimization.”, Fabian Pedregosa, Remi Leblond, and Simon Lacoste-Julien. Advances in Neural Information Processing Systems (NIPS) 2017.