GE Research &Development Center

______________________________________________________________

Technical Information Series

Applied Neural Networks for PredictingApproximate Structural ResponseBehavior Using Learning and DesignExperience

S Nagendra, J Laflen, and A Wafa

97CRD117, September 1997Class 1

Copyright 1997 General Electric Company. All rights reserved.

Corporate Research and Development

Technical Report Abstract Page

Title Applied Neural Networks for Predicting Approximate Structural Response BehaviorUsing Learning and Design Experience

Author(s) S Nagendra Phone (518)387-6280J Laflen 8*833-6280A Wafa

Component Engineering Mechanics Laboratory

ReportNumber 97CRD117 Date September 1997

Numberof Pages 11 Class 1

Key Words neural networks, turbine blades, frequency assessment, learning, design, historicaltraining

Neural networks are a class of synergistic computational paradigms that can be distinguished fromothers by their inherent fine grain parallelism, distributed adaptation, and biological inspiration. Neuralnetworks offer solutions to problems that are very difficult to solve using traditional algorithmicdecomposition techniques. The potential benefits of neural nets are the ability to learn from interactionwith the environment (e.g., in-situ data acquisition), few restrictions on the functional relationships (e.g.,inverse problems), an inherent ability to generalize training information to similar situations (e.g.,response surface approximations), and inherently parallel design and load distribution (e.g., multi-modalresponse prediction). While the initial motivation for developing artificial neural nets was to createcomputer models that could imitate certain brain functions, neural nets can be thought of as another wayof developing response surfaces. In the present study all the above aspects of neural nets are estimatedand evaluated in a practical industry application of estimating the multi-modal frequencies of differentfamilies of aircraft engine turbine blades.

Manuscript received August 7, 1997

1Applied Neural Networks for Predicting Approximate StructuralResponse Behavior Using Learning and Design Experience

S. Nagendra J. Laflen A. WafaEngineering Mechanics Lab. TaCOE Engineering Mechanics Lab.GE Corporate R&D GE Aircraft Engines GE Corporate R&D

IntroductionThis report documents the design methodology for predicting turbine blade frequenciesusing neural networks [1,2] as a design tool that draws on existing design information todevelop a design response behavior prediction model. Previous work in the application ofneural nets in predicting structural engineering response has hinted at the promise ofneural nets as function approximators, optimizers, and response surface based designtools [3-7]. The main purpose of this work is to achieve peak performance on current-generation machines exploiting their super-scalar processor and fast (but limited) cachememories. This effort implements a robust Back-Propagation (BP) algorithm developedby rearranging the BP algorithm in order to work only with matrices [8,9] and thenimplementing a fast (and portable) code for matrix multiplication, resulting in a generictest bed entitled Turbine Blade Quick Frequency Prediction Testbed (TBQFPT) withembedded matrix back propagation (EMBP).A practical design problem of interest to the industry is how to use available designknowledge as well as experimental data amassed over the years to predict responsebehavior of in-production or future products. The present study uses a turbine bladeexample (Figures 1, 2) to describe and evaluate the basic design response predictionprocedure for a family of eight aircraft engines (denoted as A-H). The adopted approachutilizes design information such as the length of the blade, chord length, wall thickness,rib thickness, dovetail and shank heights, and material properties (calculated at variousspan locations) as initial variables. The neural net is trained with respect to responsequantities (e.g., modal frequencies) measured during in-situ bench tests and used as targetresponses. The trained net is then used to predict frequencies (interpolate or extrapolate)for the trained modes. Currently up to four critical modes can be evaluated by the systemoutlined in Figure 3.

Method of ApproachThis section describes the basic ingredients needed to write the given engine family datain matrix form and present it to the neural net for training is described herein. Considerthe status of neurons in layer l ( ) ( )S lnp

(Figure 4): the pattern array consists of the

physical quantities that would influence the frequency or response characteristic ofinterest. For several known patterns it can be efficiently written in matrix form as:

2( )( ) ( )( ) ( )( ) ( )

( )( ) ( )( ) ( ) ( )( )( ) ( )( ) ( )( )( )( ) ( )( ) ( ) ( )

S l

S lS l

S l

S l S l S l

S l S l S l

S l S l S l

T

T

P T

N

N

P PNP

==

=

1

211

21 1

12

22 2

1 2

1

1

1

(1)

Row p of matrix S(l) contains the status of all the neurons in layer l when a pattern p isapplied to the network. For simplicity the input patterns are similarly organized in matrixS(0).The weights and propagated errors of associated neurons can be arranged as

( ) ( )( ) ( )( ) ( ) ( )[ ]( )( ) ( )( ) ( ) ( )( )( ) ( )( ) ( )( )

( ) ( ) ( ) ( ) ( ) ( )W l w l w l w l

w l w l w lw l w l w l

w l w l w l

N

N N Ni i i

= =

1 2

11

11

11

21

21

21

1 1 1

1

1 1 1

(2)

( )( ) ( )( ) ( )( ) ( )

( )( ) ( )( ) ( ) ( )( )( ) ( )( ) ( )( )( )( ) ( )( ) ( )( )

l

ll

l

l l l

l l l

l l l

T

T

P T

N

N

P PNP

==

=

1

211

21 1

12

22 2

1 2

1

1

1

(3)

where column n of matrix W(l) contains the weights of neuron n of layer l and row p ofmatrix (l) contains the error propagated to layer l when pattern p is applied. In additiona matrix T with the same structure as S(L) is needed to contain the target values and amatrix W(l) with the same structure of W(l) to contain the weight variation. Thus thebasic neural network structure can be perceived as a multi-layer perceptron with L layersand no connections crossing a layer (this simplifies the structure of the embedded MBP).A basic formalism is introduced here to describe the network in terms of tangible physicaland mathematical quantities. [Consider italics for scalars like indices or constants (e.g. i,j, n. l. N, L), bold for vectors (e.g. x, y. w), and capital letters for matrices (e.g. A, B) ].

Neural LayersThe input of the network is indicated as layer 0; it contains no real neurons because itspurpose is to spread the design parameters to the neurons of the first hidden layer. Thehidden layers are numbered from 1 to L-1. The output layer is L. In general, the lth layercontains Nl neurons; therefore, the input layer has Nl elements and the output layer hasNL neurons (Figure 1). A neuron n in layer l is connected to all the neurons in layer l-1through several connections (exactly Nl-1), each one associated to a weight. The weightsare organized as a vector w(n)(l) (Figure. 4) with a corresponding bias vector bn(l).

3Feed-Forward PhaseSuppose that our design knowledge data consists of P patterns ( )s p Pp = 1.. where apattern in the present case can be identified as the physical quantities (e.g., blade length,material parameters etc.) that would influence the frequency of the turbine blade . If weapply a pattern ( )s p to the input, it propagates from input to output through every layer.Each layer responds with a precise pattern that we call the status of the layer. In general,

the lth layer will be in status ( )( )s lp and the output of the network will be ( )( )s Lp . Thenthe status of the nth layer can be computed with the feed-forward rule:

( )( ) ( )( ) ( ) ( ) ( )s l f s l w l b lnp p i n ni

N

i

l

= +

=

11

1

(4)( )( ) ( )( ) ( )( ) ( )( ) ( )( ) ( )( ) ( )s l f s l w l b l f s l w l b lnp p T n n p i n n

i

N

i

l

= + = +

=

1 11

1

(4a)

where f(x) is the activation function of the neuron (in the present case f(x) = tanh(x) iscurrently used).

Prediction-Response Error ComputationThe total error E is defined as the normalized squared difference (MSE) between theoutput of the network when pattern ( )s p is applied, and the corresponding desired targetresponse output vector ( )t p can be calculated as

( ) ( ) ( ) ( ) ( )( )( )E P N t s L P N t s LL p ppp

Ln

pn

p

n

N

p

P L=

=

= ==

1 121

2

11 (5)

Back-Propagating the ErrorIn order to minimize E in respect to the weights, the weights have to be retrained usingthe error function as a guide to find the minimum error. A simple method to accomplishthis is starting from a random point in the weight space and then descending step-by-steptowards a minimum of E. If the minimum is satisfactory, the algorithm stops; otherwisewe choose another random point and repeat the descent. In order to choose the rightdirection, we can compute at each step the gradient of ( )E i e E. .

. Using the steepestdescent approach, the direction of maximum growth of E would be given by E and astep size parameter denoted herein as .The basic approach is described as a pseudocode that describes individual phases of the algorithm as outlined below.

4Phase I: Feed-Forwardfor each layer l: =1 to L

for each neuron n: = 1 to N1for each pattern p: =1 to P

( )( ) ( )( ) ( )( ) ( )( )s l f s l w l b lnp p T n n= +1 (6a)Phase II: Error Computation

for each neuron in the output layer n: =1 to N Lfor each pattern p:-1 to P

( )( ) ( ) ( )( )( ) ( )( )[ ] npL

np

np

npL

P Nt s L s L=

2 12

(6b)

Phase III: Error Back-Propagationfor each layer l:=L-1 to 1

for each neuron n:=1 to Nlfor each pattern p:=1 to P

( )( ) ( )( )[ ] ( )( ) ( )( ) np np jp n jj

Nl s l l w l

l

=

+ +

=

+1 1 121

1

(6c)

Phase IV: Step Computationfor each layer l:=1 to L

for each neuron n:=1 to Nl

( ) ( )( )b l ln npp

p=

=

1

(6d)for each weight i:=1 to

Nl1

( )( ) ( )( ) ( )( )w l l s li n npp

p

ip

=

=

1

1(6e)

Phase V: Weight updatingfor each layer l:=1 to L

for each neuron n:-1 to Nl( ) ( ) ( )b l b l b lnnew nold n= +

5for each weight i:=1 to Nl1( ) ( ) ( ) ( ) ( ) ( )w l w l w li n new i n old i n, ,= + (6f)

Search Momentum and Acceleration TermDuring steepest descent a zig-zag effect is generated by a sudden change in the directionof consecutive gradients, to proceed quicker towards the minimum, which should besmoothed [10,11]. One simple way to accomplish this is to remember the direction ofthe last step and use it to modify the direction of the current one as

( ) ( ) ( ) ( ) b l l b lnnew npp

p

nold

= +=

1 (7)

( ) ( ) ( ) ( ) ( )( ) ( ) ( ) w l l s l w li n new npp

p

ip

in old, ,

= +=

1

1(8)

The parameter regulates the influence of the previous steps on the current one. Eventhough it can heavily modify the behavior of the algorithm, for practical reasons it is oftenset to a fixed value (usually 0.9).Current optimization methods for error minimization are reliable, but their convergenceto the minimum error is slow. It is essential to have quick convergence. Based on thework performed in reference [10], a very simple improvement was implemented. Themain purpose of the acceleration technique is to vary the learning step and the momentumduring the learning in order to adapt them on-line to the shape of the error surface.The technique can be briefly summarized as follows:

(a) ( ) ( ) 0 00 0= =;(b) if ( ) ( )E t E t< 1 then ( ) ( ) ( ) t t t= =1 0;

if ( ) ( ) ( )E t E t< + 1 1 then ( ) ( ) ( ) t t t= =1 0; (9)if ( ) ( ) ( )E t E t> + 1 1 then discard the last step; ( ) ( ) ( ) t t t= =1 0;

where t is the iteration counter > 1 is the acceleration factor, < 1 is the decelerationfactor, and 0.01 0.05. If the error decreases, the learning step is increased by afactor (because the direction is correct and we want to go faster) and the momentum isretained (because it will aid the convergence). Otherwise, if the step produces an errorgreater than the previous value (a few percent more), the learning rate is decreased by afactor (for the reasons described before) and the momentum is discarded (because itwould be more misleading than beneficial). If the error increased only a few percent, thestep is accepted; some experience shows that this may be advantageous in escaping froma local minimum. The present method is quite robust in respect to the starting step;however, if the initial step is too large, some iterations are wasted at the beginning of thelearning until a good is found. On the other hand, this technique is quite sensitive to

6the acceleration and deceleration parameters as they can heavily influence theconvergence speed.

Amplified Acceleration Technique (AAT)The notion of the amplified acceleration [12] is that instead of setting the searchacceleration and deceleration factors to fixed values, we change them at every learningstep using variable amplification factors

( ) ( ) ( ) ( ) tK

K tt

KK t

a

a

d

d= +

+ =

+ 1

1 1and

(10)

where Ka and Kd are two constants (respectively for pattern learning acceleration anddeceleration). AAT has the same number of parameters as the original accelerationtechnique. The TBQFPT procedure contains an improvement of the accelerationtechnique based on references [14,15,16]. In the acceleration phase, if ( ) t 1 is small

( )( ) t Ka 1 , then ( ) ( ) t t 1 .The heuristic behind this is the following: we want to increase rapidly if it is verysmall, but we want to be cautious if is very large. The deceleration phase shows theopposite behavior: if ( ) t Kd >>1 then ( ) t Kd ; in other words the learning step isimmediately (and drastically) shortened. If ( ) t Kd

7Thus different weight functions are arrived at for each mode and a good correlation isachieved between the bench frequencies (from A-H) and the neural net prediction. Themaximum error is observed in predicting the normalized axial frequencies of about4.26%. The predictions of the neural net are well within acceptable error limits from apractical perspective.

ConclusionThe neural network test bed (TBQFPT) provides a basic test bed for estimatingfrequencies based on available design information. Design information from analysis aswell as experiments can be used to capture the design details as well as designpeculiarities as a design pattern. A robust neural net is trained for the design patternagainst the available observed frequency modes. The neural nets are then validated andthe validated set of weights is used to predict the frequency of different engines. Goodcorrelation (within 5%) was observed using the established procedure between observedbench frequencies and the neural net predictions. This indicates that neural nets can betrained to predict approximate structural response functions and can be extremely usefulin preliminary design.

Planned Future ResearchThe basic outlined approach would be enhanced to include higher modes and new designdata and evaluated for blisks (blade + disk combinations) in the near future. In addition toaircraft engine turbine blades, industrial turbine blades would also be included as part ofthe design data. Correlation with commercially available neural nets like MATLAB [17]is planned.

AcknowledgementsThe present work is carried out as part of the design productivity enhancement initiativein GE Aircraft Engines. The first and third authors would like to thank GE CRD for thetime required to work on this particular aspect through the design productivity initiative.

8Table 1. Mode: Normalized Flexural Frequencies

FAMILY BENCH NN-PREDA 0.3402 0.3401B 0.6954 0.7004C 1.0000 0.9479D 0.6876 0.7085E 0.0577 0.0571F 0.0164 0.0142G 0.4103 0.4094H 0.0000 0.0011

ERROR % Max : 3.52 Min : 0.001

Table 2. Mode: Normalized Axial Frequencies

FAMILY BENCH NN-PREDA 0.3454 0.3452B 0.7668 0.7767C 1.0000 0.9294D 0.7442 0.7809E 0.0464 0.0466F 0.0000 -0.0042G 0.4169 0.4042H 0.0206 0.0232

ERROR % Max : 4.26 Min : 0.006

Table 3. Mode: Normalized Torsional Frequencies

FAMILY BENCH NN-PREDA 0.3934 0.3966B 0.9820 0.9816C 1.0000 0.9862D 0.9871 0.9793E 0.1988 0.1908F 0.1854 0.1976G 0.4010 0.4082H 0.0000 -0.0091

ERROR % Max : 1.43 Min : 0.02

Table 4. Mode: Normalized Two Stripe Frequencies

FAMILY BENCH NN-PREDA 0.1486 0.1489B 0.3560 0.3569C 1.0000 0.9654D 0.5107 0.5207E 0.0000 -0.0014F 0.3310 0.3310G 0.6003 0.5996H 0.2381 0.2384

ERROR % Max : 1.61 Min : 0.01

9Figure 1. Design Airfoil Schematic

Figure 2. Design Blade Schematic.

10

P R E L IM IN A R YD E S IG N

P A R A M E T E R S

T A R G E T F R E Q U E N C IE S

A X IA L

F L E X U R A L

T O R S IO N A L

2 S T R IP E

N e u ra l N e t - 1

N e u ra l N e t - 2

N e u ra l N e t - 3

N e u ra l N e t - 4

1 2 3 4 5 6

O

Figure 3. Neural Net Testbed: TBQFPT Schematic

S )(p)(0

w k( )( )1

w k( )( )2

w k( )( )3

w kN( )( )

b k( )( )1

b k( )( )3

b kN

( )( )

w L( )( )1

w L( )( )2

w L( )( )3

w LN

( )( )

b L( )( )1

b L( )( )3

b LN

( )( )

S L)(p)(

Layer 1 Layer k Layer L

Figure 4: A Feed-Forward Neural Network (Multi-Layer)

11

REFERENCES[1] Rumelhart, D.E., McClelland, J., and the PDP research group, Parallel Distributed

Processing, Vol. I and II, The MIT Press, 1986.[2] Anderson, J., and Rosenfield, E., Neurocomputing : Foundations of Research, MIT

Press, Cambridge, MA 1986.[3] Vanluchene, R.D., and Sun, R., Neural Networks in Structural Engineering,

Microcomputers in Civil Engineering, Vol. 5, No. 3, pp. 207-215, 1990.[4] Hajela, P., and Berke, L, Neurobiological Computational Models in Structural

Analysis and Design, AIAA-90-1133-CP, 31st SDM Conf. Baltimore, MD, April8-10, pp. 335-343, 1991.

[5] Lee, H., and Hajela, P., Estimating MFN Trainability for Predicting TurbinePerformance, Advances in Engineering Software, Vol. 27 No. 1/2, pp. 129-136,October/November 1996, Elsevier Applied Science.

[6] Lee, H., and Hajela, L., Prediction of Turbine Performance Using MFN byReducing Mapping Nonlinearity, Artificial Intelligence and Object OrientedApproaches for Structural Engineering, Edited by G.H.V. Topping and M.Papadrakakis, pp. 99 - 105, 1995.

[7] Lee, H., and Hajela, P., On A Unified Geometrical Interpretation of MultilayerFeedforward Networks, Part II: Quantifying Mapping Nonlinearity, the 1994World Congress on Neural Networks, San Diego, CA. June 4-9, 1994.

[8] Anguita, D., Parodi, G., and Zunnio, R., Speed improvement of the Back-Propagation on current-generation workstations, WCNN '93, July 11-15, 1993,Portland, USA, pp. 165-168.

[9] Anguita, D., Parodi, G., and Zunino, R., An efficient implementation of BP onRISC-based workstations,. Neurocomputing (in press).

[10] Vogl, T.P., Mangis, J.K., Rigler, A.K., Zinik, W.T., and Alkon, D.L., Acceleratingthe Convergence of the Back-Propagation Method, Biological Cybernetics, Vol.59, pp. 257-263, 1993.0

[11] Anguita, D., Pampolini, M., Parodi, G., and Zunino, R., YPROP: Yet AnotherAccelerating Technique for the Back Propagation, ICANN '93, September 13-16,1993, Amsterdam, The Netherlands, p. 500.

[12] Hertz, J., Krough, A., and Palmer, R.G., Introduction to the Theory of NeuralComputation, Addison-Wesley, 1991.

[13] Hecht-Nielsen, R., Neurocomputing. Addison-Wesley, 1991.[14] Battiti, R., First- and Second-Order Methods for Learning: Between Steepest

Descent and Newton's Method, Neural Computation, Vol. 4, pp. 141-166, 1992.[15] Jervis, T.T., and Fitzgerald, W.J., Optimization Schemes for Neural Networks,

CUED/FINFENG/TR 144, Cambridge University Engineering Department.[16] Fahlman, S.E., An empirical study of learning speed in back-propagation

networks, CMU-CS-88-162, Carnegie Mellon University.[17] The Math Works, MATLAB V5, 24 Prime Way, Natick, MA 01760-1500, USA.

S Nagendra Applied Neural Networks for Predicting Approximate Structural Response 97CRD117J Laflen Behavior Using Learning and Design Experience September 1997A Wafa