Nonlinear EquationsChEn 2450

Nonlinear Eqns. - OverviewCharacteristics:

• May have 0 ... many solutions • Solution methods are iterative and require an initial

guess for the solution. • Not guaranteed to find the solution, even if one exists! ‣ Initial guess can be critical to finding the solution.

‣Bad initial guess may lead to no convergence, or convergence to a wrong (unintended) root.

• Solve for roots, f(x)=0. If you want f(x)=a, then write in residual form, r(x)=f(x)-a and solve r(x)=0.

Solution Approaches • Closed-Domain Methods ‣Bracket the root and “home in” on it.

‣Quite simple & robust, but require you to bound the root.

‣Can be problematic if you bound multiple roots...

• Open Domain Methods ‣Require an initial guess for the solution, but not a bracket.

‣More efficient, but less robust than closed-domain methods.

Write the equation in residual form f(x)=0.

Hoffman §3.1-3.2

Closed-domain methods

• Bisection • Regula Falsi

Yes

Hoffman §3.3

Open-domain methods

• Secant Method • Newton’s Method

No

Hoffman §3.4

Is the root bracketed, a<x<b?

Closed-Domain Methods

Bisection & Regula-Falsi for Single Nonlinear Equations

Bisection A Simple Closed-Domain Solution Method

Algorithm Given a and b, which bracket a root of f(x)

1. Set c=(a+b)/2 2. If f(a)f(c)<0 then b=c else a=c.

3. Check for convergence. If converged, return (a+b)/2. Otherwise, go to step 1.

Concept: make new guess for solution at the midpoint of the bounds.

Solving f(x)=0. So when f(x) is “close enough” to zero, we are done. • “Close enough” depends on the nominal values of f(x). • Example: if f(x) varies from 0 to 10,000, then a tolerance of 1 or 10 may be sufficient.

If f(x) varies from 0 to 10-3, then we may require a tolerance of 10-6 or smaller...

Hoffman §3.3.1

ï3 ï2 ï1 0 1 2 3ï1

ï0.5

0

0.5

1

x

f(x)

f(x)abc

Bisection - MATLAB Codefunction x = bisect( fname, a, b, tol )!% function x = bisect( fname, a, b, tol )!%!% fname - the name of the function we are solving!% a - lower bound of interval containing the root!% b - upper bound of interval containing the root!% tol - solution tolerance (absolute)!!err = tol+1; % set the initial error!!while err>tol! fa = feval(fname,a); % evaluate f(a)! c = 0.5*(a+b); % calculate c! fc = feval(fname,c); % evaluate f(c)! err = abs(fc);! if fc*fa<0! % f(c) is of opposite sign from f(a).! % This means that a and c now bracket! % the root. Set up for the next pass.! b = c;! else! % f(c) is of same sign as f(a)! % so b and c bracket the root.! % Set up for next pass.! a = c;! end!end! !% set the return value!x = 0.5*(a+b);

ï3 ï2 ï1 0 1 2 3ï1

ï0.5

0

0.5

1

x

f(x)

f(x)abc

MATLAB TipWhen writing a nonlinear equation solver, we must evaluate the function repeatedly.

function x = myEqnSolver( fname, xguess, tol )!fx = feval(fname,xguess);

Name of the function to solve. The function must

take ONE input argument.

Initial guess for the solution.

Solution tolerance - when to stop iterating?Evaluates the

function “fname” at the value “xguess”

What if we have a function in the form f( x, a, b, c ) = 0 ?

f(x) = tanh�x

a

⇥+ exp

⇤� (x + b)2

c

⌅

function y = myFunction( x, a, b, c )!!if( max(size(a))>1 | max(size(b))>1 | max(size(c))>1 )! error('Invalid parameters.')!end!y = ( tanh(x/a) + exp(-(x+b).^2/c) );

clear; clc;!tmpFun = @(x)( myFunction(x,0.1,0.6,0.12) );!xroot = myEqnSolver( tmpFun, 0.2, 1e-5 );!

Defines an “anonymous” function that takes a single argument. Calls through to

“myFunction” passing additional arguments.

Solve “tmpFun” using “myEqnSolver” with initial guess of 0.2 and tolerance of 1e-5.

Saved in a file called “myFunction.m”

Also see notes on “functions” posted on the wiki page.

Regula-Falsi “Method of False Position” - Closed Domain Method

Concept: use a line connecting the brackets to obtain the next guess (where the line crosses the x-axis).

Hoffman §3.3.2

Algorithm Given a and b, which bracket a root of f(x)

1. Calculate the slope of the line connecting

f(a) and f(b):

2. Calculate the point where this line

crosses the x-axis:

3. If f(a) f(c) < 0 then b=c else a=c.

4. Check for convergence. If converged, return c. Otherwise, go to step 1.

c = b� f(b)m

m =f(b)� f(a)

b� a

ï3 ï2 ï1 0 1 2 3ï1

ï0.5

0

0.5

1

x

f(x)

f(x)abc

Open Domain MethodsSecant & Newton’s Methods for Single Nonlinear Equations

Secant Method Open Domain Method

Hoffman §3.4.3

Algorithm Given a and b

1. Calculate the slope of the line connecting

f(a) and f(b):

2. Calculate the point where this line

crosses the x-axis:

3. Set a = b, b = c.

4. Check for convergence. If converged, return c. Otherwise, go to step 1.

c = b� f(b)m

m =f(b)� f(a)

b� a

Concept: Given two guesses, use the secant line to approximate the next guess.

Can get in trouble at local maxima/minima.

ï3 ï2 ï1 0 1 2 3ï1

ï0.5

0

0.5

1

x

f(x)

f(x)abc

−2 −1 0 1 2 3−5

0

5

10

15

20

x

f(x)

f(x)0th1st2nd3rd

Taylor Series approximations to ex at x0=1.

Newton’s Method (Open Domain Method)

Concept: use information about the function’s derivative to guide choice for next iteration.

How to get the function derivative? • Analytically. The user provides the analytic

derivative.

• Numerically. We use the function itself to estimate the derivative (more later).

Taylor Series Expansion:

Gives an approximation to f(x) using values and derivatives of f(x) at x0.

We have a guess for x (x0). We want f(x)=0.

f(x) ⇥ f(x0) +11!

f �(x0) (x� x0)Take only first two terms

of the Taylor series. (if x-x0 is small, then this

is accurate).

x = x0 �f(x0)f �(x0)

Newton’s Method

f(x) = f(x0)

+11!

f �(x0) (x� x0)

+12!

f ��(x0) (x� x0)2

+13!

f ���(x0) (x� x0)3 + · · ·

Hoffman §3.4.2

Newton’s Method AlgorithmAlgorithm

Given x0, f(x), and f ′(x) 1. Calculate f(x0) and f ′(x0) 2. Calculate new iterate for x:

3. Check for convergence. If converged, return x. Otherwise x0=x and go to step 1.

x = x0 �f(x0)f �(x0)

User-supplied Ingredients: 1. An initial guess, x0.

2. The convergence critera.

3. A function to calculate f(x). 4. A function to calculate f ′(x).

function x = newton( fname, fdername, xo, tol )!% x = newton( fname, fdername, xo, tol )!%!% Solves the nonlinear function given to find its zeros.!%!% INPUTS:!% fname - Name of the function to solve. f(x)!% fdername - Name of the function to calculate df/dx.!% This should accept a single argument, x.!% xo - The initial guess for the solution.!% tol - The solution tolerance.!%!% OUTPUT:!% The value x such that f(x)=0.

y =x3

2+ 5x2 + 2

Single Nonlinear Equations in Matlab & Excel

Single Nonlinear Equations - Excel

Define a cell for x.

Define a cell to calculate f(x). Use Goal Seek (Tools→Goal Seek) • Choose the value you want to set the

cell to (0)

• Choose the cell that you want to change (x)

Example: find the roots of y = 3x2 � 2x + 1

Example: find the roots of Use starting guess of 0.1, 0.35, 0.36, 0.75

f(x) = sin(10x3) exp��x2

⇥Example: find the roots of y = 3x2 � 2x� 1

Single Nonlinear Equations - MATLAB

fzero(‘fun’,xo) fzero(@fun,xo) fzero(f,xo) • Looks for the root (zero) of fun near xo.

• fun refers to a function that takes a value x, returns the function value, f(x). • xo is the starting guess for the solver. • Can use this for nonlinear regression (could also use for linear regression...)

Steps to Solve a Nonlinear Equation 1. Define the function you are to solve & write it in residual form f(x)=0.

2. Write a matlab function to calculate the value of f(x) given x.

3. Choose an initial guess as best as you can.

4. Use fzero to solve the problem.

use for m-file functions

use for built in functions

use for anonymous functions

Functions with Parametersy = xa function y = myfun(a,x)!

y = x.^a;What value of x gives

y=5 when a=-1.2?

clear; clc; close all;!a = -1.2;! !% create an anonymous function using!% myfun with a=-1.2. Set up for use!% with a solver to determine where it!% is equal to 5.!tmpfun = @(x)( myfun(a,x) - 5 );!!% determine the solution!xroot = fzero(tmpfun,1.0);!!% plot the results!x = logspace(-2,1);!loglog(x,myfun(a,x),'k-', ...! xroot,myfun(a,xroot),'ro', ...! x,x*0.0+5,'r-');!!% put the value of the root on the plot!text(xroot,10,strcat(‘x='num2str(xroot)))

1. Create an “anonymous function”

• tmpfun=@(x) (myfun(-1.2,x)-5)

• Creates a new function called “tmpfun” that is only a function of x. This function calls myfun with a=-1.2.

2. Use fzero on this new function

• xroot=fzero(tmpfun,1.0);

0 0.5 1 1.5 20

1

2

3

4

5

6

time (s)

heig

ht (m

)

Functions with Parametersh = h0 + v0t + 1

2at2 function h = height_accelerate( ho, vo, a, t )!h = ho + vo*t + 0.5*a*t.^2;

clear; clc; close all;! !ho = 0.0; % initial height (m)!vo = 10.0; % initial velocity (m/s)!a = -9.8; % acceleration m/s^2!h = 1.52; % find t when we are at this h! !% Create a temporary function to use with!% fsolve. We need to make a function that!% has time as its only argument.!hsolve=@(t)(height_accelerate(ho,vo,a,t)-h);! !t1 = fzero( hsolve, 1.0 );!t2 = fzero( hsolve, 3.0 );! !% set the polynomial coefficients and!% solve for roots of the polynomial!tt = roots( [a/2,vo,ho-h] )

r(t) = h0 � h + v0t + 12at2

General Function: find t to make r(t)=0.

p(t) = h0 � h + v0t + 12at2

Polynomial: find t to make p(t)=0.

ï6 ï4 ï2 0 2ï40

ï20

0

20

40

60

80

x

x3+5*x2

Minimizing & Maximizing Functions

Example: find the minimum of y = 3x2 � 2x + 1

• Minima & maxima occur in functions where the slope changes sign (i.e. where the slope is zero).

• Local vs. Global min & max.

• Polynomials: we can find all min & max (global & local)

• General functions: iterative procedure; may only find local min/max...

Nonlinear functions may have zero to many minima and maxima.

Min & Max of Functions - Excel

1. Define a cell containing the independent variable (x)

2. Define a cell containing the function value at x, f(x).

3. Choose Tools→Solver

4. Select the target cell to be f(x). 5. Set “By Changing Cells” to be x.

6. Choose either max or min

7. Click “solve”

NOTE: You can also use solver to solve a nonlinear equation (choose to set target cell to a value rather than min/max).

Min & Max of Functions - MATLAB

Minimization 1. Define a MATLAB function to evaluate f(x) given x.

2. Obtain the minimum using fmin=fminsearch(fun,x0)

Maximization 1. Define a MATLAB function to evaluate −f(x) given x.

2. Obtain the minimum using fmax=fminsearch(fun,x0)

Systems of Nonlinear Equations

Newton’s Method

Nonlinear Systems - Newton’s Method

f(x) = 0

J =

�

⇧⇧⇧⇧⇤

�f1(x)�x1

�f1(x)�x2

· · · �f1(x)�xn

�f2(x)�x1

�f2(x)�x2

· · · �f2(x)�xn

......

. . ....

�fn(x)�x1

�fn(x)�x2

· · · �fn(x)�xn

⇥

⌃⌃⌃⌃⌅

Example: n=2 equations:

f(x) � f(x0) + J�x

We want solution at f(x)=0. Therefore:

Taylor Series expansion of fi in terms of x:

This is a linear system of equations!

Newton’s Method

1. Calculate [J] & f(x0) 2. Solve for (Δx) 3. Update xi

4. If not converged, go to 1.

Algorithm Given f(x), x0, J.

x is a vector of unknowns.

Δx is a vector of corrections (updates).

J�x = �f(x0)

Hoffman §3.7

f1(x1, x2) = f1(x1,0, x2,0) +⇥f1

⇥x1(x1 � x1,0) +

⇥f1

⇥x2(x2 � x2,0)

f2(x1, x2) = f2(x1,0, x2,0) +⇥f2

⇥x1(x1 � x1,0) +

⇥f2

⇥x2(x2 � x2,0)

�

⇧⇧⇧⇤

f1(x1, x2, · · · , xn)f2(x1, x2, · · · , xn)

...fn(x1, x2, · · · , xn)

⇥

⌃⌃⌃⌅=

�

⇧⇧⇧⇤

00...0

⇥

⌃⌃⌃⌅

f

i

(x) ⇡ f

i

(x0) +nX

j=1

@f

i

@x

j|{z}Jij

(xj

� x

j0| {z }�xj

)

J - “Jacobian” matrix

Newton’s Method - ExampleOriginal Equations:

Modified Equations:

Jacobian:

12x3 + y = 4x

y = sin(x) exp(�x)

f1 =12x3 + y � 4x

f2 = y � sin(x) exp(�x)

[J ] =

⇤�f1�x

�f1�y

�f2�x

�f2�y

⌅=

�32x2 � 4 1

� cos(x) exp(�x) + sin(x) exp(�x) 1

⇥

Example - cont’d

k x y

0 -2 -4

1 -1.5183 -4.9635

2 -1.4631 -4.2932

3 -1.4611 -4.2848

4 -1.4611 -4.2848

x = �2, y = �4

x = �1.5183, y = �4.9635

1. Guess xi:

2. Calculate [J] & (f)

3. Solve for (∆)

4. Update xi

[J ] =⇤

2.0 1�3.6439 1

⌅(f) =

�0.0

2.7188

⇥

(�) =�

0.4817�0.9635

⇥

f1 =12x3 + y � 4x

f2 = y � sin(x) exp(�x)

Tips - Nonlinear SystemsBEFORE going to the computer, formulate the problem by hand!

• Define f(x)=0 and the Jacobian matrix.

• What tolerance will you use? What initial guess will you use?

• Carry out a few iterations if you can.

When implementing in MATLAB: • Write a function to calculate f(x). This should take only ONE argument: x. ‣ For a nonlinear system, this argument is a vector. Clearly define which elements of the vector

are interpreted as which variables.

• If using an analytic Jacobian, write a function to calculate J as a matrix. The function should take one argument: x.

• Write your Newton solver as a function. It should take arguments: ‣ Function name - the name of the function f(x). You can evaluate this using feval.

‣ Jacobian function name. You can evaluate this using feval.

‣ Initial guess for the solution

‣ Solution tolerance.

Example 2

y = x

3 � x + 1x

2 + y

2 = 1

f1(x1, x2) = x

21 + x

22 � 1

f2(x1, x2) = x

31 � x1 + 1� x2

J =

"�f1�x1

�f1�x2

�f2�x1

�f2�x2

#=

2x1 2x2

3x

21 � 1 �1

�

Residual form:

Jacobian Matrix:

Example 3

5x1 � x2 = 2

x1 + 4x2 = 3

Solve the linear system:

using Newton’s method.

f1(x1, x2) = 2� 5x1 + x2

f2(x1, x2) = 3� x1 � 4x2

J =

"�f1�x1

�f1�x2

�f2�x1

�f2�x2

#=

�5 1�1 �4

�

x1 = x1,0 +�x1

x2 = x2,0 +�x2

−1 0 1 2 3−10

−8

−6

−4

−2

0

2

4

6

8

x2

x1

the update...

(�x) = � [J ]�1 (f(x))

✓�x1

�x2

◆= �

�5 1�1 �4

��1 ✓f(x1,0)f(x2,0)

◆

If we choose an initial guess of [0,0]

✓x1

x2

◆=

✓0.52380.6190

◆

✓�x1

�x2

◆= �

�5 �11 �4

��1 ✓23

◆

=

✓0.52380.6190

◆

Systems of Nonlinear Equations in Matlab & Excel

Excel: use “solver” Matlab: use “fsolve” • only available if you have the optimization toolbox