There are also systematic, derivati

There are also systematic, derivative-free methods of searching for a function optimum thatresemble in some respects the algorithms that we will examine in the next section. The downhillsimplex (and other simplex) methods12 have been found to be very fast and effective for someproblems. A recent entry in the econometrics literature is the method of simulated annealing.13These derivative-free methods, particularly the latter, are often very effective in problems withmany variables in the objective function, but they usually require far more function evaluationsthan the methods based on derivatives that are considered below. Because the problems typicallyanalyzed in econometrics involve relatively few parameters but often quite complex functionsinvolving large numbers of terms in a summation, on balance, the gradient methods are usuallygoing to be preferable.