Report

Nelder Mead Fminsearch uses Nelder Mead • Fminsearch finds minimum of a function of several variables starting from an initial value. • Unconstraint nonlinear optimization method, meaning we cannot give upper or lower bounds for parameters • Downhill simplex method • Global optimization method (finds global minimum) Nelder Mead • The Nelder–Mead technique is a heuristic search method that can converge to non-stationary points. But it is easy to use and will converge for a large class of problems. • The Nelder–Mead technique was proposed by John Nelder & Roger Mead (1965). • The method uses the concept of a simplex, which is a special polynomium type with N + 1 vertices in N dimensions. • Examples of simplexes include a line segment on a line, a triangle on a plane, a tetrahedron in threedimensional space and so forth. The Nelder-Mead Algorithm • Given n+1 vertices xi, i=1… n+1 and associated function values f(xi). • Define the following coefficients – R=1 (reflection) – K=0.5 (contraction) – E=2 (expansion) – S=0.5 (shrinkage) The Nelder-Mead Algorithm 1. Sort by function value: Order the vertices to satisfy f1 < f2 < … < fn+1 2. Calculate xm = sum xi (the average of all the points except the worst) 3. Reflection. Compute xr = xm + R(xm-xn+1) and evaluate f(xr). If f1 < fr < fn accept xr and terminate the iteration. The Nelder-Mead Algorithm 4. Expansion. If fr < f1 calculate xe = xm+ K (xr xm) and evaluate f(xe). If fe < fr, accept xe; otherwise accept xr. Terminate the iteration. The Nelder-Mead Algorithm 5. Contraction. If fr > fn, perform a contraction between xm and the better of xr and xn+1. a) Outside. If fn < fr < fn+1 calculate xoc= xm+ K (xr - xm) and evaluate f(xoc). If foc< fr, accept xoc and terminate the iteration; otherwise do a shrink. b) Inside. If fr > fn+1 calculate xic = xm – K (xm- xn+1) and evaluate f(xic). If fic< fn+1 accept xic and terminate the iteration; otherwise do a shrink. The Nelder-Mead Algorithm 6. Shrink. Evaluate f at the n points vi = xi + S (xix1), i = 2,….,n+1. The vertices of the simplex at the next iteration are x1, v2, …, vn+1. Standard NM moves in 2d Example Nelder-Mead Algorithm Parameter 2 Example Nelder-Mead Algorithm Parameter 1