Support different ways to store the Hessian and Jacobian (apart of COO format)

[frapac]

Currently, MOI supports only passing the Jacobian and the Hessian in COO format:
https://github.com/jump-dev/MathOptInterface.jl/blob/master/src/nlp.jl#L123-L134
https://github.com/jump-dev/MathOptInterface.jl/blob/master/src/nlp.jl#L149-L157

Would it be possible to extend MOI to support other formats for the Jacobian/Hessian?

• Dense format
• Sparse CSC/CSR format

One idea would be to extend MOI.jacobian_structure to return the structure required by the current evaluator:

abstract type AbstractMatrixCallbackStructure end 

struct CSCStructure{VI} <: AbstractMatrixCallbackStructure
    rowval::VI
    colptr::VI
end

struct COOStructure{VI} <: AbstractMatrixCallbackStructure
    rows::VI
    cols::VI
end

struct DenseStructure <: AbstractMatrixCallbackFormat end

jacobian_structure(d::AbstractNLPEvaluator)::MOI.AbstractMatrixCallbackStructure

[blegat]

Some solvers may only implement one specific structure. E.g. Ipopt only supports COO structure for both jacobian and hessian. The user on the other hand want to describe only one structure and not have to change the structure depending on the solver.
If the structure was part of the type of the function, e.g., OracleScalarNonlinearFunction{JacobianStructure, HessianStructure}, a bridge could transform a constraint using the pair structure chosen by the user into a constraint using the pari of constraint supporting by the solver.

Another advantage of this: suppose you want to implement a bridge that create a OracleScalarNonlinearFunction-in-set constraint. Think for instance of the bridge from ScalarAffineFunction, ScalarQuadraticFunction or ExpressionScalarNonlinearFunction (AD) into OracleScalarNonlinearFunction. One could implement different version from different sparse format as different bridges. The bridge selection would then automatically select the appropriate bridge depending on what the solver supports as it does not need any bridge for transforming the structure so the bridge path length is 1 (instead of 2 if an incompatible sparsity structure is chosen). This would be important if using directly the right structure and not having to convert has noticeable impact on performance.

[frapac]

That's a great idea, I think a bridge is a good candidate for this.

• At the oracle level, we can support Structure <: Union{DenseStructure, COOStructure, CSCStructure} to specify the structure of the Jacobian and the Hessian.
• Then, if the solver does not support the specified Structure, we use a bridge to transform the constraint.

As far as I know, the different solvers support:

• Ipopt: COOStructure
• NLOpt/Optim.jl: DenseStructure
• Knitro: COOStructure, DenseStructure.
• MadNLP: COOStructure, DenseStructure (we may want to implement CSCStructure in MadNLP in the future)

Another question about the COOStructure: what do we do with duplicate entries? And shall we force them to be LowerTriangular or UpperTriangular?