constrained optimization in R

Question

I am trying to use http://rss.acs.unt.edu/Rdoc/library/stats/html/constrOptim.html in R to do optimization in R with some given linear constraints but not able to figure out how to set up the problem.

For example, I need to maximize $f(x,y) = log(x) + \frac{x^2}{y^2}$ subject to constraints $g_1(x,y) = x+y < 1$, $g_2(x,y) = x > 0$ and $g_3(x,y) = y > 0$. How do I do this in R? This is just a hypothetical example. Do not worry about its structure, instead I am interested to know how to set this up in R.

thanks!

Solution

Setting up the function was trivial:

fr <- function(x) {      x1 <- x[1]
    x2 <- x[2]
    -(log(x1) + x1^2/x2^2)  # need negative since constrOptim is a minimization routine
}

Setting up the constraint matrix was problematic due to a lack of much documentation, and I resorted to experimentation. The help page says "The feasible region is defined by ui %*% theta - ci >= 0". So I tested and this seemed to "work":

> rbind(c(-1,-1),c(1,0), c(0,1) ) %*% c(0.99,0.001) -c(-1,0, 0)
      [,1]
[1,] 0.009
[2,] 0.990
[3,] 0.001

So I put in a row for each constraint/boundary:

constrOptim(c(0.99,0.001), fr, NULL, ui=rbind(c(-1,-1),  # the -x-y > -1
                                              c(1,0),    # the x > 0
                                              c(0,1) ),  # the y > 0
                                           ci=c(-1,0, 0)) # the thresholds

For this problem there is a potential difficulty in that for all values of x the function goes to Inf as y -> 0. I do get a max around x=.95 and y=0 even when I push the starting values out to the "corner", but I'm somewhat suspicious that this is not the true maximum which I would have guessed was in the "corner". EDIT: Pursuing this I reasoned that the gradient might provide additional "direction" and added a gradient function:

grr <- function(x) { ## Gradient of 'fr'
    x1 <- x[1]
    x2 <- x[2]
    c(-(1/x[1] + 2 * x[1]/x[2]^2),
       2 * x[1]^2 /x[2]^3 )
}

This did "steer" the optimization a bit closer to the c(.999..., 0) corner, instead of moving away from it, as it did for some starting values. I remain somewhat disappointed that the process seems to "head for the cliff" when the starting values are close to the center of the feasible region:

 constrOptim(c(0.99,0.001), fr, grr, ui=rbind(c(-1,-1),  # the -x-y > -1
                                               c(1,0),    # the x > 0
                                               c(0,1) ),  # the y > 0
                                            ci=c(-1,0, 0) )
$par
[1]  9.900007e-01 -3.542673e-16

$value
[1] -7.80924e+30

$counts
function gradient 
    2001       37 

$convergence
[1] 11

$message
[1] "Objective function increased at outer iteration 2"

$outer.iterations
[1] 2

$barrier.value
[1] NaN

Note: Hans Werner Borchers posted a better example on R-Help that succeeded in getting the corner values by setting the constraint slightly away from the edge:

> constrOptim(c(0.25,0.25), fr, NULL, 
              ui=rbind( c(-1,-1), c(1,0),   c(0,1) ),  
              ci=c(-1, 0.0001, 0.0001)) 
$par
[1] 0.9999 0.0001

Comments

[Chase]

@G and others - can someone explain why cross posting is frowned upon? Would it be acceptable to mention that you are cross posting with a link? I don't have strong feelings one way or another, but some clarity on the issue is probably warranted. If this is something that the R-community at large has dealt with previously, I guess linking to those discussions would be a good start.

[IRTFM]

If you do a search on the Meta tab for "cross posting" you find a variety of opinion, most of it relatively accepting toward cross-posting. (Simultaneous cross-posting, however, seems to annoy most people.) There is a strong anti-cross-posting ethic on the r-help and cousin groups that is laid down by the R-Help Posting Guide. I have a hard time seeing the Posting Guide as being probative in SO.

[Dason]

@Chase When people cross post and don't let anybody know that then it's very possible for somebody to write up a solution for a problem that is already solved which can be a waste of their time. I personally don't care if people cross posts as long as they make it known (and it's not excessive).

[IRTFM]

In case anyone needs to see the answer on Rhelp from the crosspost it's here: stat.ethz.ch/pipermail/r-help/2011-March/273099.html

[user236215]

This is very helpful. The example I posted was not the ideal, but I can see how to set up the function. What is a feasible region?

[IRTFM]

A feasible region is the set of points that satisfies all constraints. A triangle in your specification.