Nonlinear least-squares minimization with Levenberg-Marquardt (`pwkit.lmmin`)¶

pwkit.lmmin - Pythonic, Numpy-based Levenberg-Marquardt least-squares minimizer

Basic usage:

from pwkit.lmmin import Problem, ResidualProblem

def yfunc(params, vals):
    vals[:] = {stuff with params}
def jfunc(params, jac):
    jac[i,j] = {deriv of val[j] w.r.t. params[i]}
    # i.e. jac[i] = {deriv of val wrt params[i]}

p = Problem(npar, nout, yfunc, jfunc=None)
solution = p.solve(guess)

p2 = Problem()
p2.set_npar(npar) # enables configuration of parameter meta-info
p2.set_func(nout, yfunc, jfunc)

Main Solution properties:

prob - The Problem. status - Set of strings; presence of ‘ftol’, ‘gtol’, or ‘xtol’ suggests success. params - Final parameter values. perror - 1σ uncertainties on params. covar - Covariance matrix of parameters. fnorm - Final norm of function output. fvec - Final vector of function outputs. fjac - Final Jacobian matrix of d(fvec)/d(params).

Automatic least-squares model-fitting (subtracts “observed” Y values and multiplies by inverse errors):

def yrfunc(params, modelyvalues):
modelyvalues[:] = {stuff with params}

def yjfunc(params, modelyjac):
jac[i,j] = {deriv of modelyvalue[j] w.r.t. params[i]}

p.set_residual_func(yobs, errinv, yrfunc, jrfunc, reckless=False) p = ResidualProblem(npar, yobs, errinv, yrfunc, jrfunc=None, reckless=False)

Parameter meta-information:

p.p_value(paramindex, value, fixed=False) p.p_limit(paramindex, lower=-inf, upper=+inf) p.p_step(paramindex, stepsize, maxstep=info, isrel=False) p.p_side(paramindex, sidedness) # one of ‘auto’, ‘pos’, ‘neg’, ‘two’ p.p_tie(paramindex, tiefunc) # pval = tiefunc(params)

solve() status codes:

Solution.status is a set of strings. The presence of a string in the set means that the specified condition was active when the iteration terminated. Multiple conditions may contribute to ending the iteration. The algorithm likely did not converge correctly if none of ‘ftol’, ‘xtol’, or ‘gtol’ are in status upon termination.

‘ftol’ (MINPACK/MPFIT equiv: 1, 3): “Termination occurs when both the actual and predicted relative reductions in the sum of squares are at most FTOL. Therefore, FTOL measures the relative error desired in the sum of squares.”
‘xtol’ (MINPACK/MPFIT equiv: 2, 3): “Termination occurs when the relative error between two consecutive iterates is at most XTOL. Therefore, XTOL measures the relative error desired in the approximate solution.”
‘gtol’ (MINPACK/MPFIT equiv: 4): “Termination occurs when the cosine of the angle between fvec and any column of the jacobian is at most GTOL in absolute value. Therefore, GTOL measures the orthogonality desired between the function vector and the columns of the jacobian.”
‘maxiter’ (MINPACK/MPFIT equiv: 5): Number of iterations exceeds maxiter.
‘feps’ (MINPACK/MPFIT equiv: 6): “ftol is too small. no further reduction in the sum of squares is possible.”
‘xeps’ (MINPACK/MPFIT equiv: 7): “xtol is too small. no further improvement in the approximate solution x is possible.”
‘geps’ (MINPACK/MPFIT equiv: 8): “gtol is too small. fvec is orthogonal to the columns of the jacobian to machine precision.”

(This docstring contains only usage information. For important information regarding provenance, license, and academic references, see comments in the module source code.)

class pwkit.lmmin.Problem(npar=None, nout=None, yfunc=None, jfunc=None, solclass=<class 'pwkit.lmmin.Solution'>)[source]¶

A Levenberg-Marquardt problem to be solved. Attributes:

damp: Tanh damping factor of extreme function values.
debug_calls: If true, information about function calls is printed.
debug_jac: If true, information about jacobian calls is printed.
diag: Scale factors for parameter derivatives, used to condition the problem.
epsilon: The floating-point epsilon value, used to determine step sizes in automatic Jacobian computation.
factor: The step bound is factor times the initial value times diag.
ftol: The relative error desired in the sum of squares.
gtol: The orthogonality desired between the function vector and the columns of the Jacobian.
maxiter: The maximum number of iterations allowed.
normfunc: A function to compute the norm of a vector.
solclass: A factory for Solution instances.
xtol: The relative error desired in the approximate solution.

Methods:

copy: Duplicate this Problem.
get_ndof: Get the number of degrees of freedom in the problem.
get_nfree: Get the number of free parameters (fixed/tied/etc pars are not free).
p_value: Set the initial or fixed value of a parameter.
p_limit: Set limits on parameter values.
p_step: Set the stepsize for a parameter.
p_side: Set the sidedness with which auto-derivatives are computed for a par.
p_tie: Set a parameter to be a function of other parameters.
set_func: Set the function to be optimized.
set_npar: Set the number of parameters; allows p_* to be called.
set_residual_func: Set the function to a standard model-fitting style.
solve: Run the algorithm.
solve_scipy: Run the algorithm using the Scipy implementation (for testing).

p_side(idx, sidedness)[source]¶: Acceptable values for sidedness are “auto”, “pos”, “neg”, and “two”.

class pwkit.lmmin.Solution(prob)[source]¶

A parameter solution from the Levenberg-Marquard algorithm. Attributes:

ndof - The number of degrees of freedom in the problem. prob - The Problem. status - A set of strings indicating which stop condition(s) arose. niter - The number of iterations needed to obtain the solution. perror - The 1σ errors on the final parameters. params - The final best-fit parameters. covar - The covariance of the function parameters. fnorm - The final function norm. fvec - The final function outputs. fjac - The final Jacobian. nfev - The number of function evaluations needed to obtain the solution. njev - The number of Jacobian evaluations needed to obtain the solution.

The presence of ‘ftol’, ‘gtol’, or ‘xtol’ in status suggests success.

Nonlinear least-squares minimization with Levenberg-Marquardt (pwkit.lmmin)¶

Nonlinear least-squares minimization with Levenberg-Marquardt (`pwkit.lmmin`)¶