# solverMLE -- Maximum likelihood estimate of a loopless mixed graph

## Synopsis

• Usage:
solverMLE(G,U)
• Inputs:
• G, an instance of the type Graph, or an instance of the type Digraph, or an instance of the type Bigraph, or an instance of the type MixedGraph
• U, , or a list of sample data. Alternatively, the sample covariance matrix can be given as input by setting SampleData to false
• Optional inputs:
• ChooseSolver => ..., default value EigenSolver, choose numerical solver
• ConcentrationMatrix => ..., default value false, output MLE for concentration matrix instead of MLE for covariance matrix
• DoSaturate => ..., default value true, remove saturation
• OptionsEigenSolver => ..., default value new OptionTable from {Strategy => Stickelberger, Tolerance => .000001, Basis => null, Multiplier => 0}, use options of "zeroDimSolve" in "EigenSolver"
• OptionsNAG4M2 => ..., default value new OptionTable from {EndZoneFactor => null, maxCorrSteps => null, Predictor => null, stepIncreaseFactor => null, Precision => null, tStep => null, tDegree => null, CorrectorTolerance => null, InfinityThreshold => null, gamma => null, Software => null, ResidualTolerance => null, numberSuccessesBeforeIncrease => null, Normalize => null, SingularConditionNumber => null, Projectivize => null, PostProcess => true, tStepMin => null}, use options of "solveSystem" in "NumericalAlgebraicGeometry"
• RealPrecision => ..., default value 6, number of decimals used to round input data in RR to data in QQ
• SampleData => ..., default value true, input sample covariance matrix instead of sample data
• SaturateOptions => ..., default value new OptionTable from {Strategy => null, BasisElementLimit => infinity, DegreeLimit => {}, MinimalGenerators => true, PairLimit => infinity}, use options from "saturate"
• ZeroTolerance => ..., default value 1e-10, optional input to set the largest absolute value that should be treated as zero
• Outputs:
• , consisting of (CC,Matrix,ZZ) or (CC,List,ZZ) where the first element is always a real number - the maximum value attained in the log-likelihood function, the matrix (or list of matrices) is the MLE for the covariance matrix and the integer is the ML-degree of the model. Alternatively, the MLE for the concentration matrix can be given as output by setting ConcentrationMatrix to true

## Description

This function takes as input a Graph, or a Digraph, or a Bigraph or a MixedGraph and a list or matrix that encodes, by default, the sample data. It computes the critical points of the score equations and selects the maximum value achieved among those that lie in the cone of positive-definite matrices. The default output is the maximum value in the log-likelihood function, maximum likelihood estimators (MLE) for the covariance matrix and the ML-degree of the model. MLE for the concentration matrix can be obtained by setting the optional input ConcentrationMatrix to false.

The same optional inputs as in scoreEquations can be used, plus extra optional inputs related to the numerical solver (EigenSolver by default, NAG4M2 alternatively) and its functionalities.

Below we reproduce Example 2.1.13 for the 4-cycle in the book: Mathias Drton, Bernd Sturmfels and Seth Sullivant, Lectures on Algebraic Statistics, Oberwolfach Seminars, Vol 40, Birkhauser, Basel, 2009.

 i1 : G=graph{{1,2},{2,3},{3,4},{1,4}}; i2 : U =matrix{{1,2,1,-1},{2,1,3,0},{-1, 0, 1, 1},{-5, 3, 4, -6}}; 4 4 o2 : Matrix ZZ <--- ZZ i3 : solverMLE(G,U) o3 = (-6.2615, | 7.1875 -1.625 -2.22576 5.875 |, 5) | -1.625 1.25 .875 -1.62833 | | -2.22576 .875 1.6875 -2.625 | | 5.875 -1.62833 -2.625 7.25 | o3 : Sequence

The data sample can also be given as a list:

 i4 : G=graph{{1,2},{2,3},{3,4},{1,4}}; i5 : U = {{1,2,1,-1},{2,1,3,0},{-1, 0, 1, 1},{-5, 3, 4, -6}}; i6 : solverMLE(G,U) o6 = (-6.2615, | 7.1875 -1.625 -2.22576 5.875 |, 5) | -1.625 1.25 .875 -1.62833 | | -2.22576 .875 1.6875 -2.625 | | 5.875 -1.62833 -2.625 7.25 | o6 : Sequence

In the following example we compute the MLE for the covariance matrix of the graphical model associated to the graph $1\rightarrow 2,1\rightarrow 3,2\rightarrow 3,3\rightarrow 4,3<-> 4$ In this case we give as input the sample covariance matrix:

 i7 : G = mixedGraph(digraph {{1,3},{2,4}},bigraph {{3,4}}); i8 : S = matrix {{7/20, 13/50, -3/50, -19/100}, {13/50, 73/100, -7/100, -9/100},{-3/50, -7/100, 2/5, 3/50}, {-19/100, -9/100, 3/50, 59/100}}; 4 4 o8 : Matrix QQ <--- QQ i9 : solverMLE(G,S,SampleData=>false) o9 = (-.665426, | .35 .26 -.0362724 -.19 |, 5) | .26 .73 -.07 -.145523 | | -.0362724 -.07 .4 .06 | | -.19 -.145523 .06 .59 | o9 : Sequence

Next we provide the MLE for the concentration matrix of the graphical model associated to the graph $1\rightarrow 3,2\rightarrow 4,3<->4,1 - 2$. Again the sample covariance matrix is given as input.

 i10 : G = mixedGraph(digraph {{1,3},{2,4}},bigraph {{3,4}},graph {{1,2}}); i11 : S = matrix {{7/20, 13/50, -3/50, -19/100}, {13/50, 73/100, -7/100, -9/100},{-3/50, -7/100, 2/5, 3/50}, {-19/100, -9/100, 3/50, 59/100}}; 4 4 o11 : Matrix QQ <--- QQ i12 : solverMLE(G,S,SampleData=>false,ConcentrationMatrix=>true) o12 = (-.665426, | 4.47695 -1.37472 0 1.10266 |, 5) | -1.37472 1.87907 .204176 0 | | 0 .204176 2.56734 -.210726 | | 1.10266 0 -.210726 2.07144 | o12 : Sequence

Application to positive definite matrix completion problems

Consider the following symmetric matrix with some unknown entries:

 i13 : R=QQ[x,y]; i14 : M=matrix{{115,-13,x,47},{-13,5,7,y},{x,7,27,-21},{47,y,-21,29}} o14 = | 115 -13 x 47 | | -13 5 7 y | | x 7 27 -21 | | 47 y -21 29 | 4 4 o14 : Matrix R <--- R

Unknown entries correspond to non-edges of the 4-cycle. A positive definite completion of this matrix is obtained by giving values to x and y and computing the MLE for the covariance matrix in the Gaussian graphical model given by the 4-cycle. To understand which values of x and y will result in a maximum likelihood estimate, see Example 12.16 in the book: Mateusz Michalek and Bernd Sturmfels, Invitation to Nonlinear Algebra, Graduate Studies in Mathematics, Vol ???, American Mathematical Society, 2021.

 i15 : G=graph{{1,2},{2,3},{3,4},{1,4}}; i16 : V=matrix{{115,-13,-29,47},{-13,5,7,-11},{-29,7,27,-21},{47,-11,-21,29}} o16 = | 115 -13 -29 47 | | -13 5 7 -11 | | -29 7 27 -21 | | 47 -11 -21 29 | 4 4 o16 : Matrix ZZ <--- ZZ i17 : (mx,MLE,ML)=solverMLE(G,V,SampleData=>false) o17 = (-14.5793, | 115 -13 -35.6122 47 |, 5) | -13 5 7 -6.51331 | | -35.6122 7 27 -21 | | 47 -6.51331 -21 29 | o17 : Sequence

The MLE of the covariance matrix is the unique positive definite completion of the matrix M such that its inverse, namely the concentration matrix, has zero's in the entries corresponding to non-edges of the graph. Observe that all entries of V remain the same in the MLE except for those that correspond to non-edges of the graph.

## Ways to use solverMLE :

• "solverMLE(Bigraph,Digraph,Graph,List)"
• "solverMLE(Bigraph,Digraph,Graph,Matrix)"
• "solverMLE(Bigraph,Digraph,List)"
• "solverMLE(Bigraph,Digraph,Matrix)"
• "solverMLE(Bigraph,Graph,Digraph,List)"
• "solverMLE(Bigraph,Graph,Digraph,Matrix)"
• "solverMLE(Bigraph,Graph,List)"
• "solverMLE(Bigraph,Graph,Matrix)"
• "solverMLE(Bigraph,List)"
• "solverMLE(Bigraph,Matrix)"
• "solverMLE(Digraph,Bigraph,Graph,List)"
• "solverMLE(Digraph,Bigraph,Graph,Matrix)"
• "solverMLE(Digraph,Bigraph,List)"
• "solverMLE(Digraph,Bigraph,Matrix)"
• "solverMLE(Digraph,Graph,Bigraph,List)"
• "solverMLE(Digraph,Graph,Bigraph,Matrix)"
• "solverMLE(Digraph,Graph,List)"
• "solverMLE(Digraph,Graph,Matrix)"
• "solverMLE(Digraph,List)"
• "solverMLE(Digraph,Matrix)"
• "solverMLE(Graph,Bigraph,Digraph,List)"
• "solverMLE(Graph,Bigraph,Digraph,Matrix)"
• "solverMLE(Graph,Bigraph,List)"
• "solverMLE(Graph,Bigraph,Matrix)"
• "solverMLE(Graph,Digraph,Bigraph,List)"
• "solverMLE(Graph,Digraph,Bigraph,Matrix)"
• "solverMLE(Graph,Digraph,List)"
• "solverMLE(Graph,Digraph,Matrix)"
• "solverMLE(Graph,List)"
• "solverMLE(Graph,Matrix)"
• "solverMLE(List,Bigraph)"
• "solverMLE(List,Bigraph,Digraph)"
• "solverMLE(List,Bigraph,Digraph,Graph)"
• "solverMLE(List,Bigraph,Graph)"
• "solverMLE(List,Bigraph,Graph,Digraph)"
• "solverMLE(List,Digraph)"
• "solverMLE(List,Digraph,Bigraph)"
• "solverMLE(List,Digraph,Bigraph,Graph)"
• "solverMLE(List,Digraph,Graph)"
• "solverMLE(List,Digraph,Graph,Bigraph)"
• "solverMLE(List,Graph)"
• "solverMLE(List,Graph,Bigraph)"
• "solverMLE(List,Graph,Bigraph,Digraph)"
• "solverMLE(List,Graph,Digraph)"
• "solverMLE(List,Graph,Digraph,Bigraph)"
• "solverMLE(List,MixedGraph)"
• "solverMLE(Matrix,Bigraph)"
• "solverMLE(Matrix,Bigraph,Digraph)"
• "solverMLE(Matrix,Bigraph,Digraph,Graph)"
• "solverMLE(Matrix,Bigraph,Graph)"
• "solverMLE(Matrix,Bigraph,Graph,Digraph)"
• "solverMLE(Matrix,Digraph)"
• "solverMLE(Matrix,Digraph,Bigraph)"
• "solverMLE(Matrix,Digraph,Bigraph,Graph)"
• "solverMLE(Matrix,Digraph,Graph)"
• "solverMLE(Matrix,Digraph,Graph,Bigraph)"
• "solverMLE(Matrix,Graph)"
• "solverMLE(Matrix,Graph,Bigraph)"
• "solverMLE(Matrix,Graph,Bigraph,Digraph)"
• "solverMLE(Matrix,Graph,Digraph)"
• "solverMLE(Matrix,Graph,Digraph,Bigraph)"
• "solverMLE(Matrix,MixedGraph)"
• "solverMLE(MixedGraph,List)"
• "solverMLE(MixedGraph,Matrix)"

## For the programmer

The object solverMLE is .