newton Namespace Reference

Highly flexible atomic Newton() solver and Laplace() approximation. More...

Classes
struct	HessianSolveVector
	Operator (H, x) -> solve(H, x) More...
struct	jacobian_dense_t
	Methods specific for a dense hessian. More...
struct	jacobian_sparse_plus_lowrank_t
	Methods specific for a sparse plus low rank hessian. More...
struct	jacobian_sparse_t
	Methods specific for a sparse hessian. More...
struct	newton_config
	Newton configuration parameters. More...
struct	NewtonOperator
	Generalized newton solver similar to TMB R function 'newton'. More...
struct	safe_eval
struct	TagOp
	Operator to mark intermediate variables on the tape. More...

Functions
template<class Functor , class Type >
Type	Laplace (Functor &F, Eigen::Array< Type, Eigen::Dynamic, 1 > &start, newton_config cfg=newton_config())
	Tape a functor and return Laplace Approximation. More...
template<class Functor , class Type >
vector< Type >	Newton (Functor &F, Eigen::Array< Type, Eigen::Dynamic, 1 > start, newton_config cfg=newton_config())
	Tape a functor and return solution. More...
TMBad::ad_plain	Tag (const TMBad::ad_plain &x)
	Mark a variable during taping. More...
TMBad::Scalar	Tag (const TMBad::Scalar &x)
	Otherwise ignore marks.

Detailed Description

Highly flexible atomic Newton() solver and Laplace() approximation.

Supported features

• Several hessian structures: Sparse, dense and sparse plus lowrank.
• Unlimited nesting (Newton solver within newton solver within ...)
• AD to any order at runtime
• AD Hessian of Laplace approximation (because inverse subset derivatives are implemented).
• Sparse hessian of either simplicial or supernodal kind (via preprocessor flag TMBAD_SUPERNODAL).
• CHOLMOD 64 bit integer versions for very large problems (from Eigen version 3.4).
• Saddlepoint approximation

Function Documentation

Laplace()

template<class Functor , class Type >

Type newton::Laplace	(	Functor &	F,
		Eigen::Array< Type, Eigen::Dynamic, 1 > &	start,
		newton_config	cfg = newton_config()
	)

Tape a functor and return Laplace Approximation.

Can be used anywhere in a template. Inner and outer parameters are automatically detected.

Parameters

F	Function to minimize
start	Vector with initial guess
cfg	Configuration parameters for solver

Returns

Scalar with Laplace Approximation

Note

start is passed by reference and contains the inner problem mode on output

Examples:

TMBad/spa_gauss.cpp.

Definition at line 1382 of file newton.hpp.

Newton()

template<class Functor , class Type >

vector<Type> newton::Newton	(	Functor &	F,
		Eigen::Array< Type, Eigen::Dynamic, 1 >	start,
		newton_config	cfg = newton_config()
	)

Tape a functor and return solution.

Can be used anywhere in a template. Inner and outer parameters are automatically detected.

Parameters

F	Function to minimize
start	Vector with initial guess
cfg	Configuration parameters for solver

Returns

Vector with solution

Examples:

TMBad/solver.cpp.

Definition at line 1354 of file newton.hpp.

Tag()

TMBad::ad_plain newton::Tag

(

const TMBad::ad_plain &

x

)

Mark a variable during taping.

The 'sparse plus low rank' Hessian (jacobian_sparse_plus_lowrank_t) requires the user to manually mark variables used to identify the low rank contribution.

For any intermediate variable s(x) we can write the objective function f(x) as f(x,s(x)). The hessian takes the form

hessian( f(x, s(x)) ) = jac(s)^T * f''_yy * jac(s) + R(x)

where the first term is referred to as a 'lowrank contribution' and R(x) is the remainder term. The user must choose s(x) in a way such that the remainder term becomes a sparse matrix. A good heuristic is (which you'll see by expanding the remainder term):

• Choose s(x) that links to many random effects
• Choose s(x) such that hessian(s) is sparse

Example (incomplete) of use:

PARAMETER_VECTOR(x);
Type S = x.exp().sum(); // hessian(S) is sparse!
S = Tag(S);             // Mark S for lowrank contribution
Type logS = log(S);     // hessian(log(S)) is not sparse!
return logS;

Examples:

TMBad/spde_epsilon.cpp.