l25 numerical methods part 5 project questions homework review tips and tricks summary 1
DESCRIPTION
10.57 revisited 3TRANSCRIPT
![Page 1: L25 Numerical Methods part 5 Project Questions Homework Review Tips and Tricks Summary 1](https://reader036.vdocument.in/reader036/viewer/2022062905/5a4d1ad37f8b9ab05997205a/html5/thumbnails/1.jpg)
L25 Numerical Methods part 5• Project Questions• Homework• Review• Tips and Tricks• Summary
1
![Page 2: L25 Numerical Methods part 5 Project Questions Homework Review Tips and Tricks Summary 1](https://reader036.vdocument.in/reader036/viewer/2022062905/5a4d1ad37f8b9ab05997205a/html5/thumbnails/2.jpg)
H24
2
( 1) ( )
( 1) ( 1) ( 1)
( 1) ( ) ( )
( 1)( )
( 1) ( )
( 1) ( )
( ) ( + ) ( )'( )=0( ) ( ) ( ) 0
since( )
( ) 00
k k
k T k k
k k k
kk
k k
k k
f f ffdf f dd d
ddf
x x d
x x xx
x x dx d
x dc d
F
F
![Page 3: L25 Numerical Methods part 5 Project Questions Homework Review Tips and Tricks Summary 1](https://reader036.vdocument.in/reader036/viewer/2022062905/5a4d1ad37f8b9ab05997205a/html5/thumbnails/3.jpg)
10.57 revisited
3
c1 4 0.29787234 0.057143 -4.8E-10c2 8 -0.021276596 -0.11429 1.58E-11c3 6 -0.170212766 0.114286 8.84E-10
||c|| 10.7703 0.3437 0.1714 0.0000
β 0.0010 0.2487 0.0000d1 -4 -0.301946582 -0.13224 4.77E-10d2 -8 0.013128112 0.117551 -1.6E-11d3 -6 0.164101403 -0.07347 -8.8E-10
-1.24771E-10 -1.2E-10 1.85E-18{=MMULT(TRANSPOSE(C13:C15),B19:B21)}=C13*B19+C14*B20+C15*B21
( 1) ( ) 0k k c d
( 1) ( ) 0k k c d
![Page 4: L25 Numerical Methods part 5 Project Questions Homework Review Tips and Tricks Summary 1](https://reader036.vdocument.in/reader036/viewer/2022062905/5a4d1ad37f8b9ab05997205a/html5/thumbnails/4.jpg)
10.67
4
Prob10.67
Iteration 1 2 3 4 GRG
x1 1 2 4 4x2 1 0.5 2 2
c1 -4 -1 3.51E-08 -8.3E-07c2 2 -2 -5.2E-08 1.34E-06||c|| 4.5 2.2 0.0 0.0
β 0.25 7.85E-16 632.9881d1 4 2 -3.5E-08 -2.1E-05d2 -2 1.5 5.19E-08 3.15E-05
-1.22E-07 -7.6E-09 9.86E-14
α 0.25 1 5 5
xnew1 2 4 4 3.25 4xnew2 0.5 2 2 1.75 2
f (x) -5.5 -8 -8 -7.6875 -8
2 21 2 1 1 2
1 2
2 1 *
( ) 2 4 22 4 2( *) 4 2
x
f x x x x xx xf x x
x
c x
( 1) ( ) 0k k c d
( ) ( ) ( 1)
( ) ( ) ( 1) 2( )
( 1) ( )
Conjugate Gradient
( / )
k k k
k k k k
k k
search direction
step size
d c dc c
x x d
![Page 5: L25 Numerical Methods part 5 Project Questions Homework Review Tips and Tricks Summary 1](https://reader036.vdocument.in/reader036/viewer/2022062905/5a4d1ad37f8b9ab05997205a/html5/thumbnails/5.jpg)
10.72
5
Prob 10.72
Iteration 1 2 3 4 GRGx1 1 0.382978723 0.257143 -1.6E-10x2 1 -0.234042553 -0.22857 3.85E-11x3 1 0.074468085 0.142857 1.19E-10
c1 4 0.29787234 0.057143 -2.4E-10c2 8 -0.021276596 -0.11429 7.03E-11c3 6 -0.170212766 0.114286 5.52E-10||c|| 10.8 0.3 0.2 0.0
β 0.00101856 0.248726 1.26E-17d1 -4 -0.301946582 -0.13224 2.44E-10d2 -8 0.013128112 0.117551 -7E-11d3 -6 0.164101403 -0.07347 -5.5E-10
-1.24771E-10 -6.6E-11 -8.1E-18
α 0.154255 0.416748768 1.944444 5
xnew1 0.382979 0.257142857 -1.6E-10 1.06E-09 0xnew2 -0.23404 -0.228571429 3.85E-11 -3.1E-10 0xnew3 0.074468 0.142857143 1.19E-10 -2.6E-09 0
f (x) 0.053191 0.028571429 5.38E-20 1.63E-17 0
2 2 21 2 3 1 2 2 3
1 2
2 1 3
3 2 *
( ) 2 2 2 2(1,1,1)
2 2( *) 4 2 2
4 2x
f x x x x x x x
x xf x x x
x x
xx
c x
( 1) ( ) 0k k c d
![Page 6: L25 Numerical Methods part 5 Project Questions Homework Review Tips and Tricks Summary 1](https://reader036.vdocument.in/reader036/viewer/2022062905/5a4d1ad37f8b9ab05997205a/html5/thumbnails/6.jpg)
Prob 10.76
6
Prob 10.76
Iteration 1 2 3 4 GRGx1 1 3.072336 3.113395 2.937977x2 2 1.770152 0.322241 0.32624x3 3 2.069488 1.349908 0.760734x4 4 1.979566 2.12286 2.29332
c1 -1118 22.93872 38.65691 10.06746c2 124 239.4148 -51.58023 -0.341454c3 502 107.2371 99.79143 -1.66613c4 1090 -53.0963 -31.14542 4.609562||c|| 1644.8 268.6 122.8 11.2
β 0.026674 0.209012 0.00832d1 1118 6.883109 -37.21827 -10.37712d2 -124 -242.722 0.848449 0.348514d3 -502 -120.628 -125.004 0.626082d4 -1090 24.02137 36.16616 -4.308656
-0.00256 3.01E-06 -4.43E-05
α 0.00185 0.00597 0.00471 0.09420
xnew1 3.0723 3.1134 2.9380 1.9605 2.51E-10xnew2 1.7702 0.3222 0.3262 0.3591 0xnew3 2.0695 1.3499 0.7607 0.8197 0.001241xnew4 1.9796 2.1229 2.2933 1.8874 0.001241
f (x) 259.8004 44.5808 15.6172 11.0456 0.0000
2 2 4 41 2 3 4 2 3 1 4
31 2 1 4
31 2 2 3
33 4 2 3
33 4 1 4 *
( ) ( 10 ) 5( ) ( 2 ) 10( )(1,2,3,4)
2( 10 ) 40( )20( 10 ) 4( 2 )
( *) 10( ) 8( 2 )
10( ) 40( )x
f x x x x x x x x
x x x xx x x x
fx x x xx x x x
xx
c x
( 1) ( ) 0k k c d
![Page 7: L25 Numerical Methods part 5 Project Questions Homework Review Tips and Tricks Summary 1](https://reader036.vdocument.in/reader036/viewer/2022062905/5a4d1ad37f8b9ab05997205a/html5/thumbnails/7.jpg)
Conjugate Gradient
7
( ) ( )
( 1) ( ) ( )
( ) ( ) ( 1)
( ) ( ) ( 1) 2( )
( 1) ( )
Steepest Descent
Conjugate Gradient
( / )
k k
k k k
k k k
k k k k
k k
search directionstep size
search direction
step size
d cx x d
d c dc c
x x d
Proof: http://www.cs.cmu.edu/~quake-papers/painless-conjugate-gradient.pdf
![Page 8: L25 Numerical Methods part 5 Project Questions Homework Review Tips and Tricks Summary 1](https://reader036.vdocument.in/reader036/viewer/2022062905/5a4d1ad37f8b9ab05997205a/html5/thumbnails/8.jpg)
“Deflected” Steepest Descent
8
A comparison of the convergence of gradient descent with optimal step size (in green) and conjugate vector (in red) for minimizing a quadratic function associated with a given linear system. Conjugate gradient, assuming exact arithmetic, converges in at most n steps where n is the size of the matrix of the system (here n=2). Wik
http://en.wikipedia.org/wiki/Conjugate_gradient_method
![Page 9: L25 Numerical Methods part 5 Project Questions Homework Review Tips and Tricks Summary 1](https://reader036.vdocument.in/reader036/viewer/2022062905/5a4d1ad37f8b9ab05997205a/html5/thumbnails/9.jpg)
Higher Order Methods
9
1( ) ( )
( 1) ( ) ( )
1( ) ( ) ( )
( 1) ( )
( 1) ( ) ( )
( 1)
Modified Newton
Marquardt's Compromise
( ). . 0.5 2
k k
k k k
k k kk
k k
k k k
k
search directionstep size
search directionfraction or multiple
e g orstep size
d H cx x d
d H I c
x ( )k note no alphax d
![Page 10: L25 Numerical Methods part 5 Project Questions Homework Review Tips and Tricks Summary 1](https://reader036.vdocument.in/reader036/viewer/2022062905/5a4d1ad37f8b9ab05997205a/html5/thumbnails/10.jpg)
Optimization Project
• Formulating• Computer Modeling• Solving/executing• Evaluating (your “solution”)• Analyzing the sensitivity of your solution
10
![Page 11: L25 Numerical Methods part 5 Project Questions Homework Review Tips and Tricks Summary 1](https://reader036.vdocument.in/reader036/viewer/2022062905/5a4d1ad37f8b9ab05997205a/html5/thumbnails/11.jpg)
1. Tips: Formulating
11
• Functional requirements (HoQ) • Eng. Characteristics (i.e. quantifiable measures)• Identify design variables, names, symbols, units, limits• Develop Objective function• Retrieve or develop analytical formulas/models • Develop constraints (laws of nature, man & economics)• “Principal of Optimum Sloppiness”, significant figures?
![Page 12: L25 Numerical Methods part 5 Project Questions Homework Review Tips and Tricks Summary 1](https://reader036.vdocument.in/reader036/viewer/2022062905/5a4d1ad37f8b9ab05997205a/html5/thumbnails/12.jpg)
2. Tips: Computer Modeling• Pre-test custom-written code• Hand-check (w/calculator): f(x), g(x), h(x) at some x(1) • Eliminate ratios, if possible (to avoid divide by zero)• Eliminate non-differentiable functions (such as abs(), max())• Check analytical derivatives w/FD derivatives• Exploit available library routines • Scale variables and or constraints if difficulties arise
12
![Page 13: L25 Numerical Methods part 5 Project Questions Homework Review Tips and Tricks Summary 1](https://reader036.vdocument.in/reader036/viewer/2022062905/5a4d1ad37f8b9ab05997205a/html5/thumbnails/13.jpg)
3. Tips: Solving/Executing
13
• Test “optimizer” w/ known problems/solutions• Solve from multiple starting points• If algorithm fails, monitor each iteration• Record statistics: constraint values, solutions
![Page 14: L25 Numerical Methods part 5 Project Questions Homework Review Tips and Tricks Summary 1](https://reader036.vdocument.in/reader036/viewer/2022062905/5a4d1ad37f8b9ab05997205a/html5/thumbnails/14.jpg)
4. Tips: Evaluating the Solution
14
• Hand-check (calculator): f(x*), g(x*), h(x*)• Evaluate constraint activity
– Violated– Non-binding/inactive– Binding/active
• Do results make physical sense?
![Page 15: L25 Numerical Methods part 5 Project Questions Homework Review Tips and Tricks Summary 1](https://reader036.vdocument.in/reader036/viewer/2022062905/5a4d1ad37f8b9ab05997205a/html5/thumbnails/15.jpg)
5. Tips: Analyzing the sensitivity
15
• Relax R.H.S. • Record Δf(x) for Δx• Change cost coefficients in f(x)• Vary parameters in g(x), h(x)
• Remember: A, b and c’s• Look for opportunity!
![Page 16: L25 Numerical Methods part 5 Project Questions Homework Review Tips and Tricks Summary 1](https://reader036.vdocument.in/reader036/viewer/2022062905/5a4d1ad37f8b9ab05997205a/html5/thumbnails/16.jpg)
Test 5 on Wed
• T/F• Region elimination methods• Steepest descent algorithim • Conjugate Gradient algorithm• Be prepared to do hand calculations.
16
![Page 17: L25 Numerical Methods part 5 Project Questions Homework Review Tips and Tricks Summary 1](https://reader036.vdocument.in/reader036/viewer/2022062905/5a4d1ad37f8b9ab05997205a/html5/thumbnails/17.jpg)
Alternate Equal Interval
Golden Section
Equal Interval aka “Exhaustive”
Fractional Reduction
17
ln( )1ln(0.618)
FRN
Add these formulas to your notes for next test!
1/ (0.618)nnew oldFR I I
2ln( )1ln(2 / 3)
FRN
2/1new oldFR I I
n
2 1NFR
/2/ (2 / 3)nnew oldFR I I
![Page 18: L25 Numerical Methods part 5 Project Questions Homework Review Tips and Tricks Summary 1](https://reader036.vdocument.in/reader036/viewer/2022062905/5a4d1ad37f8b9ab05997205a/html5/thumbnails/18.jpg)
Summary• Steepest descent algorithm may stall• Conjugate Gradient
Convergence in n iterations (n=# of design var’s)Still has lots of Fcn evals (in line search)May need to restart after n+1 iterations
• Use “TIPS” to facilitate your project
18