Knitro / MATLAB reference

Usage of knitromatlab is described here.

What is knitromatlab?

knitromatlab is the interface used to call Knitro from the MATLAB environment.

knitromatlab’s syntax is similar to MATLAB’s built-in optimization function fmincon. The main differences are:

  • knitromatlab has additional input arguments for additional features and options.
  • There is a separate function, knitromatlab_mip, to solve mixed-integer nonlinear programs.
  • knitromatlab does not require the Optimization Toolbox.
  • Many returned flags and messages differ, because they are returned directly from Knitro libraries.

Syntax

The most elaborate form is

[x,fval,exitflag,output,lambda,grad,hessian] = ...
    knitromatlab(fun,x0,A,b,Aeq,beq,lb,ub,nonlcon,extendedFeatures,options,knitroOpts)

but the simplest function call reduces to:

x = knitromatlab(fun,x0)

Any of the following forms may be used:

x = knitromatlab(fun,x0)
x = knitromatlab(fun,x0,A,b)
x = knitromatlab(fun,x0,A,b,Aeq,beq)
x = knitromatlab(fun,x0,A,b,Aeq,beq,lb,ub)
x = knitromatlab(fun,x0,A,b,Aeq,beq,lb,ub,nonlcon)
x = knitromatlab(fun,x0,A,b,Aeq,beq,lb,ub,nonlcon,extendedFeatures)
x = knitromatlab(fun,x0,A,b,Aeq,beq,lb,ub,nonlcon,extendedFeatures,options)
x = knitromatlab(fun,x0,A,b,Aeq,beq,lb,ub,nonlcon,extendedFeatures,options,knitroOpts)
[x,fval] = knitromatlab(...)
[x,fval,exitflag] = knitromatlab(...)
[x,fval,exitflag,output] = knitromatlab(...)
[x,fval,exitflag,output,lambda,] = knitromatlab(...)
[x,fval,exitflag,output,lambda,grad] = knitromatlab(...)
[x,fval,exitflag,output,lambda,grad,hessian] = knitromatlab(...)

x = knitromatlab_mip(fun,x0,A,b,Aeq,beq,lb,ub,nonlcon,xType)
x = knitromatlab_mip(fun,x0,A,b,Aeq,beq,lb,ub,nonlcon,xType,objFnType)
x = knitromatlab_mip(fun,x0,A,b,Aeq,beq,lb,ub,nonlcon,xType,objFnType,cineqFnType)
x = knitromatlab_mip(fun,x0,A,b,Aeq,beq,lb,ub,nonlcon,xType,objFnType,cineqFnType, ...
        extendedFeatures)
x = knitromatlab_mip(fun,x0,A,b,Aeq,beq,lb,ub,nonlcon,xType,objFnType,cineqFnType, ...
        extendedFeatures,options)
x = knitromatlab_mip(fun,x0,A,b,Aeq,beq,lb,ub,nonlcon,xType,objFnType,cineqFnType, ...
        extendedFeatures,options,knitroOpts)
[x,fval] = knitromatlab_mip(...)
[x,fval,exitflag] = knitromatlab_mip(...)
[x,fval,exitflag,output] = knitromatlab_mip(...)
[x,fval,exitflag,output,lambda,] = knitromatlab_mip(...)
[x,fval,exitflag,output,lambda,grad] = knitromatlab_mip(...)
[x,fval,exitflag,output,lambda,grad,hessian] = knitromatlab_mip(...)

An additional function, knitrolink, may be used in place of the old ktrlink interface. knitrolink has the same input and output arguments as ktrlink, but it is equivalent to using knitromatlab with an empty array for the value of extendedFeatures. If you are using knitrolink, please switch to knitromatlab as the knitrolink function will be deprecated in a future release.

Input Arguments

Input Argument Description
fun

The function to be minimized.

fun accepts a vector x and returns a scalar f, the objective function evaluated at x.

If exact gradients are used, an additional vector with the objective gradient should be returned.

x0 The initial point vector.
A Linear inequality constraint coefficient matrix.
b Linear inequality constraint upper bound vector.
Aeq Linear equality constraint coefficient matrix.
beq Linear equality constraint right-hand side vector.
lb Variable lower bound vector.
ub Variable upper bound vector.
nonlcon

The function that computes the nonlinear inequality and equality constraints.

nonlcon accepts a vector x and returns two vectors, the values of the nonlinear inequality functions at x and the values of the nonlinear equality functions at x.

If exact gradients are used, two additional matrices should be returned with the gradients for the nonlinear inequality functions and the gradients for the nonlinear equality functions.

extendedFeatures

The structure used to define other, extended modeling features of Knitro.

Currently it is used for complementarity constraints, the output function, parallel finite differencing, initial lambda values, custom finite-difference step sizes, custom feasibility tolerances, custom scalings, custom treatments of integer variables, specification of linear variables, custom setting of honor bounds, and some Hessian and Jacobian information.

The two complementarity constraint fields are extendedFeatures.ccIndexList1 and extendedFeatures.ccIndexList2 which contain the variable index lists for variables complementary to each other. The same index may not appear more than once in the lists.

The lambdaInitial field allows the user to specify the initial lambda values in a structure with a different field for each constraint type. The fields are ineqlin for linear inequality constraints, eqlin for linear equality constraints, ineqnonlin for nonlinear inequality constraints, eqnonlin for nonlinear equality constraints, upper for upper bounds of variables, and lower for lower bounds of variables. If only some of the fields are defined, the missing fields will be filled with zeros. If none of the fields are defined, Knitro will compute an initial value.

The FinDiffRelStep field can be used to specify custom step size values for finite-differencing.

The LinearVars field can be used to specify which variables in the model only appear linearly in the objective and constraints. This information can then be used to perform additional presolve operations. See KTR_set_linearvars() for more details.

The HonorBnds field can be used to specify which variables in the model must satisfy their bounds throughout the optimization. See KTR_set_honorbnds() for more details.

The fields AFeasTols, AeqFeasTols, cFeasTols, ceqFeasTols, and xFeasTols can be used to specify custom feasibility tolerances for problem constraints and variables.

The fields AScaleFactors, AeqScaleFactors, cScaleFactors, ceqScaleFactors, xScaleFactors, xScaleCenters, and objScaleFactor can be used to specify custom scalings for problem constraints, variables, and the objective. See KTR_set_var_scalings(), KTR_set_con_scalings() and KTR_set_obj_scaling() for details on how these scalings should be defined.

The field xIntStrategy can be used to specify customized treatments for integer variables. See KTR_mip_set_intvar_strategy() for more details.

The other six available fields are JacobPattern, HessPattern, HessFcn, HessMult, OutputFcn, and UseParallel. They have the same properties as the options set by optimset.

options The options structure set with optimset.
knitroOpts The text file with Knitro options.

The objective function, fun, can be specified as a function handle for a MATLAB function, as in

x = knitromatlab(@objFunc,x0)

with a function

function [f,g] = objFunc(x)
    f = x^2;
    g = 2*x;

or as a function handle for an anonymous function:

x = knitromatlab(@(x)x^2,x0)

The constraint function, nonlcon, is similar, but it returns at least two vectors, c and ceq. It may additionally return two matrices, the gradient matrix of the nonlinear inequality constraints, and the gradient matrix of the nonlinear equality constraints. The third and fourth arguments are only needed when GradConstr is ‘on’ or gradopt is set to 1. See http://www.mathworks.com/help/optim/ug/nonlinear-constraints-with-gradients.html for more details.

Output Arguments

Output Argument Description
x The optimal solution vector.
fval The optimal solution objective value.
exitflag Integer identifying the reason for termination of the algorithm.
output Structure containing information about the optimization.
lambda Structure containing the Lagrange multipliers at the solution with a different field for each constraint type.
grad Gradient vector at x.
hessian Hessian matrix at x. For notes on when this option is available, see KTR_get_hessian_values().

Output Structure Fields

Output Argument Field Description
output iterations Number of iterations.
output funcCount Number of function evaluations.
output constrviolation Maximum of constraint violations.
output firstorderopt Measure of first-order optimality.
lambda lower Lower bounds
lambda upper Upper bounds
lambda ineqlin Linear inequalities
lambda eqlin Linear equalities
lambda ineqnonlin Nonlinear inequalities
lambda eqnonlin Nonlinear equalities

Setting Options

knitromatlab takes up to two options inputs. The first is in fmincon format, using optimset, and the second is a Knitro options text file. Because the full version of optimset requires a MATLAB Optimization Toolbox license, HessFcn, HessMult, HessPattern, JacPattern, OutputFcn, and UseParallel can also be used with the extendedFeatures structure. All the other options have equivalent ways of being set in the Knitro options text file. If a Knitro options text file is specified, unspecified options will still use the default option values from the fmincon format options. Settings from the Knitro options text file and the extendedFeatures structure will take precedence over settings made in the MATLAB options structure. Note that options set with the optimoptions function are not compatible with knitromatlab functions.

To use Knitro options, create an options text file as described for the callable library, and include the file as the 12th argument in the call to knitromatlab, or the 15th argument in the call to knitromatlab_mip.

Options Structure Example:

options = optimset('Algorithm', 'interior-point', 'Display','iter', ...
        'GradObj','on','GradConstr','on', ...
        'JacobPattern',Jpattern,'Hessian','user-supplied','HessPattern',Hpattern, ...
        'HessFcn',@hessfun,'MaxIter',1000, ...
        'TolX', 1e-15, 'TolFun', 1e-8, 'TolCon', 1e-8, 'UseParallel', true);
[x,fval,exitflag,output,lambda,grad,hess] = ...
        knitromatlab(@objfun,x0,A,b,Aeq,beq,lb,ub,@constfun,[],options,[]);

The example above shows how to set options using the MATLAB options structure. The example below shows how to set the same options using extendedFeatures and a Knitro options file.

Options File Example:

extendedFeatures.JacobPattern = Jpattern;
extendedFeatures.HessPattern = Hpattern;
extendedFeatures.HessFcn = @hessfun;
extendedFeatures.UseParallel = true;
[x,fval,exitflag,output,lambda,grad,hess] = ...
        knitromatlab(@objfun,x0,A,b,Aeq,beq,lb,ub,@constfun,extendedFeatures,[], ...
        'knitro.opt');

knitro.opt:

algorithm direct    # Equivalent to setting 'Algorithm' to 'interior-point'
outlev iter_verbose # Equivalent to setting 'Display' to 'iter'
gradopt exact       # Equivalent to setting 'GradObj' to 'on' and 'GradConstr' to 'on'
hessopt exact       # Equivalent to setting 'Hessian' to 'on'
maxit 1000          # Equivalent to setting 'MaxIter' to 1000
xtol 1e-15          # Equivalent to setting 'TolX' to 1e-15
opttol 1e-8         # Equivalent to setting 'TolFun' to 1e-8
feastol 1e-8        # Equivalent to setting 'TolCon' to 1e-8

Options

Option Equivalent Knitro Option Description
Algorithm algorithm The optimization algorithm: ‘interior-point’, ‘active-set’, or ‘sqp’. Default: ‘interior-point’
AlwaysHonorConstraints honorbnds Bounds are satisfied at every iteration if set to the default ‘bounds’. They are not necessarily satisfied if set to ‘none’.
DerivativeCheck derivcheck

Check the value of the user-provided exact gradients at a random point against the finite difference gradients. If the difference is not within the specified tolerance, Knitro will stop execution and display the violation. May be set to ‘off’ (default) or ‘on’.

The default relative tolerance is 1e-6, but can be changed with the derivcheck_tol option in the Knitro options file. The finite difference method is set by FinDiffType, and is set to ‘forward’ by default.

Display outlev
Level of display
  • ‘off’ or ‘none’ displays no output.
  • ‘iter’ displays information for each iteration, and gives the default exit message.
  • ‘iter-detailed’ displays information for each iteration, and gives the technical exit message.
  • ‘notify’ displays output only if the function does not converge, and gives the default exit message.
  • ‘notify-detailed’ displays output only if the function does not converge, and gives the technical exit message.
  • ‘final’ (default) displays just the final output, and gives the default exit message.
  • ‘final-detailed’ displays just the final output, and gives the technical exit message.
FinDiffType

gradopt,

derivcheck_type

The finite difference type is either ‘forward’ (default) or ‘central’. ‘central’ takes twice as many function evaluations and may violate bounds during evaluation, but is usually more accurate.

When exact derivatives are used and DerivativeCheck is used, this option sets the finite difference type to ‘forward’ (default) or ‘central’ to compare with the exact derivatives.

Option Equivalent Knitro Option Description
GradConstr gradopt
Gradient for nonlinear constraint functions.
  • ‘off’ (default) sets the algorithm to use finite differences to estimate the gradients of nonlinear constraints.
  • ‘on’ sets the algorithm to expect exact gradients of the nonlinear constraints in the third and fourth constraint function outputs, as described for nonlcon in the Input Arguments section.

To use sparse gradients, the sparsity pattern must be set with the JacobPattern option.

GradObj gradopt
Gradient for nonlinear objective function.
  • ‘off’ (default) sets the algorithm to use finite differences to estimate the gradients of the objective function.
  • ‘on’ sets the algorithm to expect exact gradients of the objective in the second objective function outputs, as decribed for fval in the Input Arguments section.
HessFcn  

Function handle to the user-supplied Hessian.

Default: []

Hessian hessopt

Sets the Hessian option for Knitro.

Default: ‘bfgs’

HessMult  

Handle to a user-supplied function that returns a Hessian-times-vector product.

Default: []

HessPattern  

Sparsity pattern of Hessian.

Default: ‘sparse(ones(n))’

InitBarrierParam bar_initmu

Initial barrier value.

Default: 0.1

InitTrustRegionRadius delta

Initial radius of the trust region.

Default: ‘sqrt(n)’

JacobPattern  

Sparsity pattern of the Jacobian of the nonlinear constraint matrix. It can be used for finite-differencing or for user-supplied gradients.

Default: ‘sparse(ones(Jrows,Jcols))’

MaxFunEvals maxfevals

Maximum number of function evaluations allowed.

Default: -1 (no limit)

MaxIter maxit

Maximum number of iterations allowed.

Default: 10000

MaxProjCGIter cg_maxit

Tolerance for the number of projected conjugate gradient iterations.

Default: ‘2*(n-numberOfEqualities)’

ObjectiveLimit objrange

Specifies the extreme limits of the objective function for purposes of determining unboundedness.

If the magnitude of the objective function becomes greater than objrange for a feasible iterate, then the problem is determined to be unbounded and Knitro proceeds no further.

Default: 1.0e20

OutputFcn newpoint KTR_set_newpt_callback

Handle to a user-supplied function that is called after every iteration of the algorithm and returns a boolean indicating if the algorithm should stop. See more details below.

Default: []

Option Equivalent Knitro Option Description
ScaleProblem scale

The default value of ‘obj-and-con’ allows Knitro to scale the objective and constraint functions based on their values at the initial point. Setting the option to ‘none’ disables scaling

If scaling is performed, all internal computations, including the stopping tests, are based on the scaled values.

SubproblemAlgorithm algorithm Determines how the iteration step is calculated. The default option is ‘ldl-factorization’, (in Knitro this is a symmetric, indefinite factorization) which is usually faster than the alternative, ‘cg’ (conjugate gradient). Conjugate gradient may be faster for large problems with dense Hessians.
TolCon feastol

Termination tolerance on the constraint violation.

Default: 1.0e-6

TolFun opttol

Termination tolerance on the function value.

Default: 1.0e-6

TolX xtol

Termination tolerance on x.

Default: 1.0e-6

UseParallel  

Boolean indicating if parallel finite differences will be used. It has no effect when exact gradients are used or if the Parallel Computing Toolbox is not installed.

The Knitro option “par_numthreads” does not have an effect on parallel finite differences in MATLAB. The MATLAB command “parpool(n)” will set the number of workers to the minimum of n and the maximum number allowed, which can be set in the cluster profile. If the parallel pool is not started before knitromatlab is run, it will start one with the default number of workers set by MATLAB, as long as the Parallel Pool preferences allow automatically creating a parallel pool.

Default: false

Output Function

The output function can be assigned with

extendedFeatures.OutputFcn = @outputfun;

or in the options structure with

options = optimset('OutputFcn',@outputfun);

where the function is defined as

function stop = outputfun(x,optimValues,state);

Only the value of stop can be set to true or false. Setting it to true will terminate Knitro.

The inputs to the function cannot be modified. The inputs include the current point x, the structure optimValues, and the state. Since Knitro only calls the function after every iteration, the value of state will always be ‘iter’. The optimValues structure contains the following fields:

optimValues Fields

optimValues Field Description
lambda Structure containing the Lagrange multipliers at the solution with a different field for each constraint type.
fval The objective value at x.
c The nonlinear inequality constraint values at x.
ceq The nonlinear equality constraint values at x.
gradient The gradient vector at x.
cineqjac The nonlinear inequality constraint Jacobian matrix.
ceqjac The nonlinear equality constraint Jacobian matrix.

Note that setting newpoint to any value other than 3 in the Knitro options file will take precedence over OutputFcn. Note that the nonlinear constraint Jacobian matrices are given with the variables as the rows and constraints as the columns, the transpose of JacobPattern.

Sparsity Pattern for Nonlinear Constraints

The sparsity pattern for the constraint Jacobian is a matrix, which is passed as the JacobPattern option. JacobPattern is only for the nonlinear constraints, with one row for each constraint. The nonlinear inequalities, in order, make up the first rows, and the nonlinear equalities, in order, are in the rows after that. Gradients for linear constraints are not included in this matrix, since the sparsity pattern is known from the linear coefficient matrices.

All that matters for the matrix is whether the values are zero or not zero for each entry. A nonzero value indicates that a value is expected from the gradient function. A MATLAB sparse matrix may be used, which may be more efficient for large sparse matrices of constraints.

The gradients of the constraints returned by the nonlinear constraint function and those used in the newpoint function have the transpose of the Jacobian pattern, i.e., JacobPattern has a row for each nonlinear constraint and a column for each variable, while the gradient matrices (one for inequalities and one for equalities) have a column for each constraint and a row for each variable.

Hessians

The Hessian is the matrix of the second derivative of the Lagrangian, as in

http://www.mathworks.com/help/optim/ug/fmincon.html#brh002z

The matrix H can be given as a full or sparse matrix of the upper triangular or whole matrix pattern.

If HessMult is used, then the Hessian-vector-product of the Hessian and a vector supplied by Knitro at that iteration is returned.

Nonlinear Least Squares

There is a special function, knitromatlab_lsqnonlin, for using Knitro to solve nonlinear least squares problems. It behaves similarly to the lsqnonlin function in the MATLAB Optimization Toolbox. Note that the extendedFeatures structure is not an input argument of lsqnonlin, but is the argument before options in knitromatlab_lsqnonlin. If the structure is not used, an empty matrix, [], should be used in its place.

The most elaborate form is:

[x,resnorm,residual,exitflag,output,lambda,jacobian] = ...
              knitromatlab_lsqnonlin(fun,x0,lb,ub,extendedFeatures,options,knitroOpts)

but the simplest function call reduces to:

x = knitromatlab_lsqnonlin(fun,x0)

Input Arguments for knitromatlab_lsqnonlin

Input Argument Description
fun

The function whose sum of squares is minimized.

fun accepts a vector x and returns a vector F, the values of the functions evaluated at x.

If exact gradients are used, an additional matrix should be returned with the Jacobian for the function at x. Unlike the user-supplied Jacobian for knitromatlab, the entries J(i,j) of the Jacobian for knitromatlab_lsqnonlin represent the partial derivative of the function component i with respect to variable j.

x0 The initial point vector.
lb Variable lower bound vector.
ub Variable upper bound vector.
extendedFeatures

The structure used to define other, extended modeling features of Knitro.

It is similar to the extendedFeatures input to knitromatlab, but currently it is only used for the JacobPattern (rows are the function components and columns are the variables) and OutputFcn fields.

options The options structure set with optimset. See details below.
knitroOpts The text file with Knitro options.

The options available in the options structure are the same as those available in knitromatlab, except the differences noted here.

  • GradConstr, HessFcn, Hessian, HessMult, and HessPattern are not available.
  • JacobPattern is the Jacobian for the function, where rows are the function components and columns are the variables.
  • OutputFcn inputs refer to the transformed problem, but x still refers to the current point.
  • Algorithm may be set to ‘interior-point’ (default) to use the Gauss-Newton method, or ‘levenberg-marquardt’ to use the Levenberg-Marquardt method.

knitromatlab_lsqnonlin does not use Hessian information or options provided by the user, but uses the approximation shown in Least squares problems.

Nonlinear System of Equations

There is another special function, knitromatlab_fsolve, for using Knitro to solve nonlinear systems of equations. It behaves similarly to the fsolve function in the MATLAB Optimization Toolbox. Note that the extendedFeatures structure is not an input argument of fsolve, but is the argument before options in knitromatlab_fsolve. If the structure is not used, an empty matrix, [], should be used in its place.

The most elaborate form is:

[x,fval,exitflag,output,jacobian] = ...
              knitromatlab_fsolve(fun,x0,extendedFeatures,options,knitroOpts)

but the simplest function call reduces to:

x = knitromatlab_fsolve(fun,x0)

Input Arguments for knitromatlab_fsolve

Input Argument Description
fun

The function whose components are to be solved to equal zero.

fun accepts a vector x and returns a vector F, the values of the functions evaluated at x.

If exact gradients are used, an additional matrix should be returned with the Jacobian for the function at x. Unlike the user-supplied Jacobian for knitromatlab, the entries J(i,j) of the Jacobian for knitromatlab_fsolve represent the partial derivative of the function component i with respect to variable j.

x0 The initial point vector.
extendedFeatures

The structure used to define other, extended modeling features of Knitro.

It is similar to the extendedFeatures input to knitromatlab, but currently it is only used for the JacobPattern (rows are the function components and columns are the variables) and OutputFcn fields.

options The options structure set with optimset. See details below.
knitroOpts The text file with Knitro options.

The options available in the options structure are the same as those available in knitromatlab, except the differences noted here.

  • GradObj, HessFcn, HessMult, and HessPattern are not available. Hessian is available, except when using the ‘levenberg-marquardt’ algorithm option, but may only be set to ‘bfgs’ (default), ‘lbfgs’, or ‘fin-diff-grads’.
  • JacobPattern is the Jacobian for the function, where rows are the function components and columns are the variables.
  • OutputFcn inputs refer to the transformed problem, but x still refers to the current point.
  • In addition to the four standard Knitro algorithm options, Algorithm may also be set to ‘trust-region-dogleg’ (default), or ‘levenberg-marquardt’. Using ‘trust-region-dogleg’ is equivalent to using ‘interior-point’ with ‘SubproblemMethod’ set to ‘cg’ (Knitro algorithm 2). Using ‘levenberg-marquardt’ is equivalent to using knitromatlab_lsqnonlin.

Note on exit flags

The returned exit flags will correspond with Knitro’s return code, rather than matching fmincon’s exit flags.

Return codes

Upon completion, Knitro displays a message and returns an exit code to MATLAB. In the example above Knitro found a solution, so the message was:

Locally optimal solution found

with the return value of exitflag set to 0.

If a solution is not found, then Knitro returns one of the following:

Value Description
0 Locally optimal solution found.
-100 Current feasible solution estimate cannot be improved. Nearly optimal.
-101 Relative change in feasible solution estimate < xtol.
-102 Current feasible solution estimate cannot be improved.
-103 Relative change in feasible objective < ftol for ftol_iters.
-200 Convergence to an infeasible point. Problem may be locally infeasible.
-201 Relative change in infeasible solution estimate < xtol.
-202 Current infeasible solution estimate cannot be improved.
-203 Multistart: No primal feasible point found.
-204 Problem determined to be infeasible with respect to constraint bounds.
-205 Problem determined to be infeasible with respect to variable bounds.
-300 Problem appears to be unbounded.
-400 Iteration limit reached. Current point is feasible.
-401 Time limit reached. Current point is feasible.
-402 Function evaluation limit reached. Current point is feasible.
-403 MIP: All nodes have been explored. Integer feasible point found.
-404 MIP: Integer feasible point found.
-405 MIP: Subproblem solve limit reached. Integer feasible point found.
-406 MIP: Node limit reached. Integer feasible point found.
-410 Iteration limit reached. Current point is infeasible.
-411 Time limit reached. Current point is infeasible.
-412 Function evaluation limit reached. Current point is infeasible.
-413 MIP: All nodes have been explored. No integer feasible point found.
-415 MIP: Subproblem solve limit reached. No integer feasible point found.
-416 MIP: Node limit reached. No integer feasible point found.
-501 LP solver error.
-502 Evaluation error.
-503 Not enough memory.
-504 Terminated by user.
-505 to -522 Input or other API error.
-523 Derivative check failed.
-524 Derivative check finished.
-600 Internal Knitro error.

For more information on return codes, see Return codes.