discrete-time zhang neural networks for time-varying nonlinear optimizationdownloads.hindawi.com ›...

Research ArticleDiscrete-Time Zhang Neural Networks for Time-VaryingNonlinear Optimization

Min Sun 12 Maoying Tian3 and YijuWang2

1School of Mathematics and Statistics Zaozhuang University Shandong 277160 China2School of Management Qufu Normal University Shandong 276826 China3Department of Physiology Shandong Coal Mining Health School Shandong 277011 China

Correspondence should be addressed to Min Sun ziyouxiaodou163com

Received 16 January 2019 Revised 24 February 2019 Accepted 12 March 2019 Published 8 April 2019

Academic Editor Zhengqiu Zhang

Copyright copy 2019 Min Sun et alThis is an open access article distributed under the Creative Commons Attribution License whichpermits unrestricted use distribution and reproduction in any medium provided the original work is properly cited

As a special kind of recurrent neural networks Zhang neural network (ZNN) has been successfully applied to various time-variantproblems solving In this paper we present three Zhang et al discretization (ZeaD) formulas including a special two-step ZeaDformula a general two-step ZeaD formula and a general five-step ZeaD formula and prove that the special and general two-stepZeaD formulas are convergent while the general five-step ZeaD formula is not zero-stable and thus is divergent Then to solve thetime-varying nonlinear optimization (TVNO) in real time based on the Taylor series expansion and the above two convergent two-step ZeaD formulas we discrete the continuous-time ZNN (CTZNN)model of TVNO and thus get a special two-step discrete-timeZNN (DTZNN) model and a general two-step DTZNN model Theoretical analyses indicate that the sequence generated by thefirst DTZNN model is divergent while the sequence generated by the second DTZNN model is convergent Furthermore for thestep-size of the second DTZNNmodel its tight upper bound and the optimal step-size are also discussed Finally some numericalresults and comparisons are provided and analyzed to substantiate the efficacy of the proposed DTZNNmodels

1 Introduction

As a subcase of nonlinear programming nonlinear opti-mization has been widely encountered in a variety of sci-entific and engineering applications and many applicationscan be modeled or reformulated as nonlinear optimizationeg the Markowitz mean-variance model in finance thetransportation problem in management the shortest pathin network model the (non)convex separable optimizationin image denoising etc [1 2] Due to its fundamentalroles nonlinear optimization has been extensively studied bymany researchers during the last several centuries and manyefficient algorithms have been developed and investigatedin the literature [3ndash7] which can be classified into twocategoriesThe first category includes the first-order iterationmethods which only use the first derivative information ofthe objective function such as the steepest descent methodthe conjugate gradient method and the memory gradientmethod These methods are suitable to solve large-scalenonlinear optimization due to their simple structure and low

storage Numerically the conjugate gradient method and thememory gradient method usually perform better than thesteepest descent method [6] The second category includesthe Newton method and its variants eg the quasi-NewtonBFGS and DFP methods which need to compute the secondderivative of the objective function or an approximation of itTherefore they are not suitable to solve large scale nonlinearoptimization though they possess locally fast convergentrate Then to overcome the drawback of the quasi-Newtonmethod Nocedal [7] designed a limited memory BFGSmethod (L-BFGS) for nonlinear optimization and numericalresults indicate that the L-BFGS method is very efficient dueto its low storage In addition during the last decades neuralnetwork has drawn extensive attention of researchers andpractitioners due to its nice properties eg distributed stor-age high-speed parallel-processing hardware applicationsand superior performance in large-scale online applications[8] Some neural networks have been developed to solvenonlinear optimization during the last decades [9ndash11]

HindawiDiscrete Dynamics in Nature and SocietyVolume 2019 Article ID 4745759 14 pageshttpsdoiorg10115520194745759

2 Discrete Dynamics in Nature and Society

Table 1 ZeaD formulas used in [12 15ndash17]

Formula 119899-step General form Truncation error

[15] 119891119896 = 119891119896+1 minus 119891119896120591 1 No 119874(1205912)[12] 119891119896 = 2119891119896+1 minus 3119891119896 + 2119891119896minus1 minus 119891119896minus22120591 3 No 119874(1205913)[12] 119891119896 = 6119891119896+1 minus 3119891119896 minus 2119891119896minus1 minus 119891119896minus210120591 3 No 119874(1205913)[17] 119891119896 = (minus1198861 + 12)119891119896+1 + 31198861119891119896 minus (31198861 + 12)119891119896minus1 + 1198861119891119896minus2120591 3 Yes 119874(1205913)[16] 119891119896 = (1198861 + 13)119891119896+1 minus (41198861 minus 12)119891119896 + (61198861 minus 1)119891119896minus1120591

minus (41198861 minus 16)119891119896minus2 minus 1198861119891119896minus3120591 4 Yes 119874(1205914)

Most of the above algorithms are designed intrinsicallyfor solving static nonlinear optimization (SNO) thereforethey might not be effective enough for solving time-varyingnonlinear optimization (TVNO) whose objective functiondenoted by 119891(119909(119905) 119905) is a multivariate function with respectto the decision variable 119909 and the time variable 119905 Theconnection between SNO and TVNO is obvious (1) SNO isa special case of TVNO and when 119909(119905) = 119909 for any 119905 ge 0TVNO reduces to SNO (2) the discrete-time form of TVNOwhose objective function is 119891(119909119896 119905119896) (119905119896 = 119896120591 119896 = 0 1 )and 120591 gt 0 denoting the sampling gap can be viewed asa sequence of SNO At each single time instant 119905 = 119905119896TVNO can be viewed as a static nonlinear optimizationand consequently it can be solved by the above mentionedalgorithms However this treatment is not advisable due tothe following three reasons (1) it is usually inefficient andlowly precise due to it need to solve a sequence of SNO [12](2) in the online solution process of discrete-time TVNOthe present andor previous data with respect to 119909119894 (119894 le 119896)should be used sufficiently to generate the unknown decisionvariable 119909119896+1 (3) most importantly at time instant 119905119896 we donot know the future information such as the function values119891(119909(119905119896+1) 119905119896+1) and only the current and past informationfor time instances 119905119895 with 119895 le 119896 can be used Therefore theconventional static methods and the neural networks in [9ndash11] which are based on the future information cannot solveTVNO

As a special recurrent neural network Zhang neuralnetwork (ZNN) named after Chinese scholar ZhangYunongserves as a unified approach to solve various online time-varying problems such as time-varying quadratic functionminimization [13] future minimization [14] time-varyingmatrix pseudoinversion [8] and TVNO [12 15 16] Forexample based on ZNN Jin et al [15] presented a one-stepdiscrete-time ZNN (DTZNN)model for TVNO whosemax-imal residual error is theoretically O(1205912) Subsequently Guoet al [12] proposed two DTZNN models for TVNO whichbelong to three-step DTZNN with steady-state residual error(SSRE) changing in an O(1205913) manner Then quite recently

Zhang et al [16] presented a general four-step discrete-timederivative dynamics model and a general four-step DTZNNmodel for TVNO both models contain a free parameterwhich can take any values of the interval (112 16) and isconvergent with truncation error of O(1205914)

Generally speaking in the real-time solution processof TVNO the more the past data is utilized the smallerthe truncation error of the corresponding Zhang et aldiscretization (ZeaD) formula is For example the truncationerrors of the one-step DTZNN model in [15] the three-step DTZNN models in [12] the general three-step DTZNNmodels in [17] and the four-step DTZNN models in [16]are O(1205912) O(1205913) O(1205913) and O(1205914) respectively The corre-sponding ZeaD formulas are summarized in Table 1 in whichthe effective domains of the parameter 1198861 in [17] and [16] are(minusinfin 0) and (112 16) respectively Obviously the generalZeaD formula with 1198861 = minus12 minus110 in [17] reduces the twospecial ZeaD formulas in [12]

In this paper we are going to further study the ZeaDformula and three ZeaD formulas are presented including aspecial two-step ZeaD formula with truncation error 119874(1205913)a general two-step ZeaD formula with truncation error119874(1205912) and a general five-step ZeaD formula with truncationerror 119874(1205915) We prove that the first two ZeaD formulasis convergent while the third ZeaD formula is not zero-stable and thus is divergent Then based on the Taylor seriesexpansion and the above two convergent ZeaD formulaswe discrete the continuous-time ZNN (CTZNN) model forTVNO and thus get a special two-step DTZNN model anda general two-step DTZNN model for TVNO Theoreticalanalyses indicate that the first DTZNN model is divergentwhile the second DTZNN model is convergent for any 1198861 isin(minus12 +infin) and the step-size ℎ isin (0 (2 + 41198861)(1 + 1198861)) Inaddition the tight upper bound of the step-size ℎ and theoptimal step-size are also discussed

The rest of the paper is organized as follows We firstrecall some basic definitions and results in Section 2 includ-ing problem formulation of TVNO continuous-time ZNN(CTZNN) model for TVNO and the general 119899-step ZeaD

Discrete Dynamics in Nature and Society 3

formula In Section 3 a special two-step ZeaD formulawith truncation error of 119874(1205913) and a general two-step ZeaDformula with truncation error of 119874(1205912) are presented andanalyzed and we also prove that the five-step ZeaD formulawith truncation error of 119874(1205915) or 119874(1205916) is divergent in thissection Furthermore based on the two convergent ZeaDformulas two discrete-time ZNN (DTZNN) models forTVNO are presented and we prove that the first DTZNNmodel is not convergent and the second DTZNN is con-vergent Later in Section 4 some numerical experimentsare presented to illustrate and compare the performances ofthe convergent two-step DTZNN model with other variantsFinally a concluding remark with future research direction isgiven in Section 5 The main contributions of this paper aresummarized as follows

(1)ThreeZeaD formulas are presented including a specialtwo-step ZeaD formula a general two-step ZeaD formulaand a general five-stepZeaD formula whose convergence andstability are discussed in detail

(2) Two DTZNN models are given to solve TVNObased on two convergent two-step ZeaD formulas whoseconvergence is also studied in detail

(3)The feasible region of the step-size ℎ in the convergentDTZNN model is studied and its tight upper bound and theoptimal step-size are also discussed

(4) The high precision of the convergent DTZNN modelis substantiated in numerical tests

2 Preliminaries

In this section the results in [12 16] are summarized forthe foundation of further discussion including the problemformulation of TVNO the CTZNN model for TVNO andthe general ZeaD formula

Firstly the problem formulation of the TVNO is asfollows [16]

min119909(119905)isinR119899

119891 (119909 (119905) 119905) isin R 119905 isin [0 119905119891] (1)

where the time-varying nonlinear function 119891(sdot sdot) R119899 times[0 119905119891] 997888rarr R is second-order differentiable and boundedProblem (1) aims to find 119909(119905) isin R119899 such that the function119891(119909(119905) 119905) achieves its minimum at any time 119905 ge 0Thus in thesequent analyse we assume that the solution of problem (1)exists at any time 119905 ge 0

It is well known that it is often hard to find the globaloptimum solution of time-invariant nonlinear optimiza-tion by traditional numerical algorithms [3 6] Thereforeresearchers have resorted to find the stationary point of time-invariant nonlinear optimizationWe also transformproblem(1) into finding its stationary point 119909(119905) which satisfies thefollowing nonlinear equations

119892 (119909 (119905) 119905) ≐ 120597119891 (119909 (119905) 119905)120597119909 (119905) = 0 isin R119899 forall119905 isin [0 119905119891] (2)

where symbol ≐ denotes the computational assignmentoperation In the following we aim to find the solutions ofproblem (2) at any time 119905 ge 0 Generally speaking the

solutions of problem (1) are the solutions of problem (2) butthe inversemay be not and if119891(119909(119905) 119905) is convex with respectto 119909(119905) the inverse also holds [3]

Setting 119890(119905) = 119892(119909(119905) 119905) in the following Zhang neuralnetwork (ZNN) design formula [18]

119890 (119905) ≐ 119889119890 (119905)119889119905 = minus120574119890 (119905) 120574 gt 0 (3)

we get the following continuous-time ZNN (CTZNN) modelof problem (1) [15]

(119905) = minus119867 (119909 (119905) 119905)minus1 (120574119892 (119909 (119905) 119905) + 119892119905 (119909 (119905) 119905)) (4)

where 119892119905(119909(119905) 119905) is the partial derivative of the mapping119892(119909(119905) 119905) with respect to its second variable 119905 ie119892119905 (119909 (119905) 119905) = 120597119892 (119909 (119905) 119905)120597119905 = 1205972119891 (119909 (119905) 119905)120597119909 (119905) 120597119905

= [ 12059721198911205971199091120597119905 12059721198911205971199092120597119905

1205972119891120597119909119899120597119905]⊤ isin R

119899(5)

and 119867(119909(119905) 119905) is the Hessian matrix of problem (1) ie

119867(119909 (119905) 119905) =

[[[[[[[[[[[[[[[

120597211989112059711990911205971199091120597211989112059711990911205971199092 sdot sdot sdot 12059721198911205971199091120597119909119899120597211989112059711990921205971199091120597211989112059711990921205971199092 sdot sdot sdot 12059721198911205971199092120597119909119899

d

1205972119891120597119909119899120597119909112059721198911205971199091198991205971199092 sdot sdot sdot 1205972119891120597119909119899120597119909119899

]]]]]]]]]]]]]]]

isin R119899times119899

(6)

which is assumed to be positive definite throughout the paperto ensure that the stationary point 119909(119905) of problem (1) is alsoits solution

Remark 1 The main difference of the CTZNN model (4)and the neural network models in [9ndash11] lies in the formerincluding the information of the time derivative 119892119905(119909(119905) 119905) toget fast and accurate solution of TVNO while the motionequation in the latter neural network models for SNO andother optimization problems such as variational inequalitiesand complementarity problems can be expressed by

(119905) = 119865 (119909) (7)

and 119865(119909) R119899 997888rarr R119899 which is a mapping with respect tothe decision variable119909 and is an implicit functionwith respectto 119905 Furthermore it generally satisfies 119865(119909lowast) = 0 where 119909lowast isa solution of problem solving When 119909(119905) = 119909 for any 119905 ge 0that is to say that TVNO only contains the decision variable119909 and thus reduces to SNO the CTZNNmodel (4) reduces to

(119905) = minus120574119867 (119909)minus1 119892 (119909) (8)


Obviously minus120574119867(119909lowast)minus1119892(119909lowast) = 0 for any 119909lowast being a solutionof SNO thus the CTZNNmodel (4) becomes a special neuralnetwork for SNO when 119909(119905) = 119909 for any 119905 ge 0

The general 119899-step ZeaD formula is defined as follows[19]

119891119896 = 1120591 (119899+1sum119894=1

119886119894119891119896minus119899+119894) + 119874 (120591119901) (9)

where 119899 is the amount of the steps of ZeaD formula (9) 119886119894 isinR (119894 = 1 2 119899 + 1) denotes the coefficients119874(120591119901) denotesthe truncation error119891119896 is the value of119891(119905) at time instant 119905119896 =119896120591 ie 119891119896 = 119891(119905119896) 119896 denotes the updating index Equation(9) with 1198861 = 0 119886119899+1 = 0 is termed as 119899-step 119901th-order ZeaDformula

3 Multistep ZeaD Formulas andDiscrete-Time Models

In this section we first propose two two-step ZeaD formulaswith truncation error of 119874(1205913) and 119874(1205912) and prove thatthe five-step ZeaD formula with truncation error of 119874(1205915)or 119874(1205916) is not convergent Then two DTZNN models forTVNO are presented and analyzed subsequently

31 Concepts of Convergence of Discrete-Time Models Thefollowing concepts about zero-stability and consistency areused to analyze the theoretical results of our proposeddiscrete-time models [20]

Concept 1 The zero-stability of an 119899-step discrete-timemethod

119909119896+1 + 119899sum119894=1

120572119894119909119896+1minus119894 = 120591 119899sum119894=0

120573119894V119896+1minus119894 (10)

can be checked by determining the roots of the characteristicpolynomial 119875(120592) = 120592119899 + sum119899119894=1 120572119896120592119899minus119894 If the roots of 119875(120592) = 0are such that

(i) all roots lie in the unit disk ie |120592| le 1(ii) any roots on the unit circle (ie |120592| = 1) are simple

(ie not multiple)

then the 119899-step discrete-time method is zero-stable

Concept 2 An 119899-step discrete-time method is said to beconsistent with order 119901 if its truncation error is O(120591119901) with119901 gt 0 for the smooth exact solution

Concept 3 For an 119899-step discrete-time method it is con-vergent ie 119909[(119905minus1199050)120591] 997888rarr 119909lowast(119905) for all 119905 isin [1199050 119905119891] as120591 997888rarr 0 if and only if such an algorithm is zero-stableand consistent (see Concepts 1 and 2) That is zero-stabilityand consistency result in convergence In particular a zero-stable and consistent method converges with the order of itstruncation error

32 Multistep ZeaD Formulas Based on the Taylor seriesexpansion

119891119896+1 = 119891119896 + 120591 119891119896 + 12059122 119891119896 + 12059133119891119896 + + 120591119898119898119891(119898)119896

+ 119874 (120591119898+1) (11)

where 119898 is a nonnegative integer we can derive the two-stepZeaD formula which is presented in the following lemma

Lemma 2 (see [21]) e two-step ZeaD formula with trunca-tion error of 119874(1205913) can be expressed as

119891119896 = 119891119896+1 minus 119891119896minus12120591 (12)

which is convergent and the general two-step ZeaD formulawith truncation error of 119874(1205912) can be expressed as

119891119896 = (1 + 1198861) 119891119896+1 minus (1 + 21198861) 119891119896 + 1198861119891119896minus1120591 (13)

which is convergent for any 1198861 isin (minus12 +infin)0 Proof The proof is presented in Appendix A

After we received the reviewersrsquo comments on the paperwe were brought to the attention of the references [21 22]which have presented the rigorous proofs of Lemma 2 andthe following Lemma 5 However for the completeness of thepaper we have decided to give the proofs in the Appendix

Remark 3 If 1198861 = 0 then the general two-step ZeaD formula(13) reduces to the one-step ZeaD formula in [15]

119909119896+1 = 119909119896 minus 119867 (119909119896 119905119896)minus1 (ℎ119892 (119909119896 119905119896) + 120591119892119905 (119909119896 119905119896)) (14)

Then in the following the effective domain of 1198861 is set as(minus12 +infin)Corollary 4 For any fixed and sufficiently small samplinggap 120591 gt 0 the truncation error of the general two-step ZeaDformula (13) is decreasing as the parameter 1198861 997888rarr minus12+Proof For any fixed and sufficiently small sampling gap 120591 gt 0from the proof of Lemma 2 the truncation error is dominatedby the term

(1198861 + 12) 1205912 119891119896 (15)

which obviously becomes smaller as 1198861 997888rarr minus12+The following theorem reveals that five-step ZeaD for-

mula with truncation error of 119874(1205915) or 119874(1205916) is not conver-gent

Lemma 5 (see [22]) Five-step ZeaD formula with truncationerror of 119874(1205915) or 119874(1205916) is not convergentProof The proof is presented in Appendix B


Remark 6 The polynomial (B15) (see Appendix B) has fiveroots which are denoted by 120574119894(1198866) (119894 = 1 2 5) Definingthe moduli maximum function as

119876 (1198866) = max 1003816100381610038161003816120574119894 (1198866)1003816100381610038161003816 | 119894 = 1 2 5 (16)

Figure 1 shows the graph of the function 119860(1198866) fromwhich we observe that 119876(1198866) is always bigger than 1 except1198866 = 0 However from the definition of the general 119899-stepZeaD formula we have that 1198866 = 033 Discrete-Time ZNN Models In this subsection twodiscrete-time ZNN (DTZNN) models are presented forTVNO based on the two convergent two-step ZeaD formulas(12) and (13)

Firstly applying the two-step 2th-order ZeaD formula(12) to discretize the CTZNNmodel (4) we get the followingDTZNNmodel for TVNO

119909119896+1 = 119909119896minus1minus 119867 (119909119896 119905119896)minus1 (ℎ119892 (119909119896 119905119896) + 2120591119892119905 (119909119896 119905119896)) (17)

where the step-size ℎ = 2120591120574 gt 0Similarly applying the general two-step 1th-order ZeaD

formula (13) to discretize the CTZNN model (4) we get thefollowing DTZNNmodel for TVNO

119909119896+1= 1 + 211988611 + 1198861 119909119896 minus

11988611 + 1198861 119909119896minus1minus 119867 (119909119896 119905119896)minus1 (ℎ119892 (119909119896 119905119896) + 1205911 + 1198861119892119905 (119909119896 119905119896))

(18)

where the step-size ℎ = 120591120574(1 + 1198861) gt 0 and 1198861 isin (minus12 +infin)Remark 7 (I) The DTZNN models (17) and (18) can also beused to solve SNO In this case 119892119905(119909119896 119905119896) = 0 and thus aspointed in [12] the DTZNN model (17) reduces to 119909119896+1 =119909119896minus1 minus ℎ119867(119909119896 119905119896)minus1119892(119909119896 119905119896) Furthermore when ℎ = 1 wehave

119909119896+1 = 119909119896minus1 minus 119867 (119909119896 119905119896)minus1 119892 (119909119896 119905119896) (19)

which is exactly the Newton algorithm for SNO [3] Further-more in this case the DTZNNmodel (18) reduces to

119909119896+1 = 1 + 211988611 + 1198861 119909119896 minus11988611 + 1198861 119909119896minus1

minus ℎ119867 (119909119896 119905119896)minus1 119892 (119909119896 119905119896) (20)

which can be viewed a two-step iterative method for SNOwhen 1198861 = minus12

(II) The iterative schemes of DTZNN models (17) and(18) are similar to those of discrete neural network (DNN)models in [9ndash11] which are obtained by discreting themotionequation of neural network (7) by the Euler method and canbe expressed by

119909119896+1 = 119866 (119909119896 119905119896) (21)

minus100 minus50 0 50 1000

5

10

15

20

25

Q

a6

Q=1Q=Q(a6)

Figure 1 The graph of the moduli maximum function 119876(1198866)

where 119866(119909) R119899 997888rarr R119899 Generally the sequence 119909119896generated by the iterative scheme (21) converges globally to asolution of the static problem solving while in the followingwe shall prove that the sequence generated by the DTZNNmodel (18) is convergent in the sense that the sequence ofsteady-state residual error (SSRE) 119892(119909119896 119905119896)2 convergesto zero with order 119874(1205912) In fact the following Theorem 9indicates that 119892(119909119896 119905119896)2 can be written as 119892(119909119896 119905119896)2 =11988811199031198961 +11988821199031198962 +119874(1205912) where max |1199031| |1199032| lt 1Therefore at thetime instant 119905119896 (for sufficiently large 119896) the generated iterate119909119896 can approximate the solution 119909lowast(119905119896) of TVNO with highprecision when the sampling gap 120591 is sufficiently small suchas 120591 = 001 0001Theorem 8 Suppose (119909119896 119905119896) is the sequence generated bythe two-step DTZNN model (17) and let 119892(119909119896 119905119896)2 be thegenerated steady-state residual error (SSRE)en the sequence119892(119909119896 119905119896)2 is divergentProof Let Δ119909119896 = 119909119896+1 minus 119909119896 and Δ119909119896minus1 = 119909119896 minus 119909119896minus1 Then theproposed two-step DTZNN model (17) can be reformulatedas

12Δ119909119896 + 12Δ119909119896minus1= minus119867 (119909119896 119905119896)minus1 (ℎ2119892 (119909119896 119905119896) + 120591119892119905 (119909119896 119905119896))

(22)

On the other hand by the Taylor series expansion we have

119892 (119909119896+1 119905119896+1) = 119892 (119909119896 119905119896) + 119867 (119909119896 119905119896) Δ119909119896+ 120591119892119905 (119909119896 119905119896) + O (1205912) (23)


and

119892 (119909119896minus1 119905119896minus1) = 119892 (119909119896 119905119896) minus 119867 (119909119896 119905119896) Δ119909119896minus1minus 120591119892119905 (119909119896 119905119896) + O (1205912) (24)

where O(Δ1199092119896) and O(Δ1199092119896minus1) are absorbed into O(1205912) as theyare assumed to be of the same order of magnitude [20] Bythe algebraic manipulation ldquo(23)2 minus (24)2rdquo the followingresults can be obtained

12119892 (119909119896+1 119905119896+1) minus 12119892 (119909119896minus1 119905119896minus1)= 12119867 (119909119896 119905119896) (Δ119909119896 + Δ119909119896minus1) + 120591119892119905 (119909119896 119905119896)

+ O (1205912) (25)

which together with (22) implies

12119892 (119909119896+1 119905119896+1) minus 12119892 (119909119896minus1 119905119896minus1)= minusℎ2119892 (119909119896 119905119896) + O (1205912)

(26)

ie

(119892 (119909119896+1 119905119896+1) minus O (1205912)) minus (119892 (119909119896minus1 119905119896minus1) minus O (1205912))= minusℎ (119892 (119909119896 119905119896) minus O (1205912)) (27)

Setting 119866119896 = 119892(119909119896+1 119905119896+1) minus O(1205912) (27) can be written as

119866119896+1 + ℎ119866119896 minus 119866119896minus1 = 0 (28)

The characteristic equation of the difference equation (28) is

1205822 + ℎ120582 minus 1 = 0 (29)

which has two different real roots from the discriminant Δ =ℎ2 + 4 gt 0 By [23] at least one root of the real quadraticequation (29) is greater than or equal to one inmodulusThusthe sequence 119866119896 is divergent so is the sequence 119892(119909119896 119905119896)The proof is completed

Theorem 9 Suppose (119909119896 119905119896) is the sequence generated bythe two-step DTZNN model (18) and let 119892(119909119896 119905119896)2 be thegenerated steady-state residual error (SSRE)en for any 1198861 isin(minus12 +infin) and the step-size ℎ isin (0 (2 + 41198861)(1 + 1198861)) wehave that lim119896997888rarrinfin119892(119909119896 119905119896)2 is of order 119874(1205912) and thus thesequence 119892(119909119896 119905119896)2 convergence of order 119874(1205912) is to zeroProof Let Δ119909119896 = 119909119896+1 minus 119909119896 and Δ119909119896minus1 = 119909119896 minus 119909119896minus1 Then thetwo-step DTZNNmodel (18) can be reformulated as

(1 + 1198861) Δ119909119896 minus 1198861Δ119909119896minus1= minus119867 (119909119896 119905119896)minus1 ((1 + 1198861) ℎ119892 (119909119896 119905119896) + 120591119892119905 (119909119896 119905119896)) (30)

By the algebraic manipulation ldquo(1 + 1198861) times (23)+1198861 times (24)rdquo weget the following equation

(1 + 1198861) 119892 (119909119896+1 119905119896+1) + 1198861119892 (119909119896minus1 119905119896minus1)= (1 + 21198861) 119892 (119909119896 119905119896)

+ 119867 (119909119896 119905119896) ((1 + 1198861) Δ119909119896 minus 1198861Δ119909119896minus1)+ 120591119892119905 (119909119896 119905119896) + O (1205912)

(31)


(1 + 1198861) 119892 (119909119896+1 119905119896+1) + 1198861119892 (119909119896minus1 119905119896minus1)= (1 + 21198861 minus (1 + 1198861) ℎ) 119892 (119909119896 119905119896) + O (1205912) (32)

ie

(119892 (119909119896+1 119905119896+1) minus O (1205912))+ 11988611 + 1198861 (119892 (119909119896minus1 119905119896minus1) minus O (1205912)) )

= 1 + 21198861 minus (1 + 1198861) ℎ1 + 1198861 (119892 (119909119896 119905119896) minus O (1205912)) )(33)

Similarly letting 119866119896 = 119892(119909119896+1 119905119896+1) minus O(1205912) (33) can bewritten as

119866119896+1 minus 1 + 21198861 minus (1 + 1198861) ℎ1 + 1198861 119866119896 + 11988611 + 1198861119866119896minus1 = 0 (34)


1205822 minus 1 + 21198861 minus (1 + 1198861) ℎ1 + 1198861 120582 + 11988611 + 1198861 = 0 (35)

By [23] two roots of (35) are less than one in modulus if andonly if

1003816100381610038161003816100381610038161003816100381611988611 + 1198861

10038161003816100381610038161003816100381610038161003816 lt 11003816100381610038161003816100381610038161003816100381610038161 + 21198861 minus (1 + 1198861) ℎ1 + 1198861

100381610038161003816100381610038161003816100381610038161003816 lt 1 + 11988611 + 1198861 (36)

Obviously the first inequality always holds for any 1198861 gt minus12therefore we only analyze the second inequality which isequivalent to10038161003816100381610038161 + 21198861 minus (1 + 1198861) ℎ1003816100381610038161003816 lt 10038161003816100381610038161 + 11988611003816100381610038161003816 + 1198861 (37)

and thus 10038161003816100381610038161 + 21198861 minus (1 + 1198861) ℎ1003816100381610038161003816 lt 1 + 21198861 (38)

So

0 lt ℎ lt 2 + 411988611 + 1198861 (39)

Then the sequence 119866119896 is convergent for any 1198861 isin(minus12 +infin) and step-size ℎ isin (0 (2 + 41198861)(1 + 1198861)) so isthe sequence 119892(119909119896 119905119896) The proof is completed


Obviously two initial states (ie1199090 1199091) are needed to startthe iteration of the DTZNNmodel (18) We use the followingDTZNNmodel [15] to initiate the iterative computation

1199091 = 1199090 minus 119867 (1199090 1199050)minus1 (ℎ119892 (1199090 1199050) + 120591119892119905 (1199090 1199050)) (40)

Remark 10 The upper bound of the step-size ℎ that is (2 +41198861)(1 + 1198861) is an increasing function with respect to theparameter 1198861 and when 1198861 997888rarr +infin it converges to 4

The following theorem shows that the upper bound of thestep-size ℎ that is (2 + 41198861)(1 + 1198861) is tightTheorem 11 For any 1198861 gt minus12 if ℎ ge (2+41198861)(1+1198861) thenthe sequence 119892(119909119896 119905119896)2 generated by the DTZNN model(18) does not converge to zero

Proof Ifℎ ge (2+41198861)(1+1198861) then the characteristic equationof the difference equation (34) reduces to

1205822 + 1 + 211988611 + 1198861 120582 + 11988611 + 1198861 = 0 (41)

which has two different real roots1205821 = minus11205822 = minus 11988611 + 1198861

(42)

Thus the general solution of the difference equation (34) is

119866119896 = 1198881 (minus1)119896 + 1198882 ( 11988611 + 1198861)119896 (43)

where 1198881 1198882 are two arbitrary constant which are determinedby two initial states 1199090 1199091 So the limit of the sequence 119866119896general does not exist except 1198881 = 0 which indicates that thesequence 119892(119909119896 119905119896)2 generally does not converge to zeroThis completes the proof

In the remainder of this subsection let us investigatethe optimal step-size for given 1198861 isin (minus12 +infin) Thediscriminant of (35) is

Δ = 1(1 + 1198861)2 (ℎ

2 minus 2 (1 + 21198861) ℎ + 1) (44)

where ℎ = (1 + 1198861)ℎ isin (0 2 + 41198861) Set Δ 1 = ℎ2 minus 2(1 +21198861)ℎ+1 which is a quadratic function with respect to ℎ andits discriminant is

Δ = 41198861 (1 + 1198861) (45)

The following analyses are divided into two cases accordingto the sign of Δ

(1) If 1198861 isin (minus12 0) Δ is less than zero and thus Δ ispositive which indicates that (35) has two different real rootswhich are denoted by 1205821 1205822 Therefore it holds that

1205821 + 1205822 = minus1 + 21198861 minus (1 + 1198861) ℎ1 + 1198861 12058211205822 = 11988611 + 1198861

(46)

and the second equation indicates that 120582119894 (119894 = 1 2) havecontrary sign then we assume that 1205821 gt 0 1205822 lt 0 withoutloss of generality Furthermore for any 1198861 isin (minus12 0) and thestep-size ℎ isin (0 (2 + 41198861)(1 + 1198861)) fromTheorem 9 we have0 le |120582119894| lt 1 (119894 = 1 2) The general solution of (34) can bewritten as

119866119896 = 11988811205821198961 + 11988821205821198962 (47)which together with (46) results in the following model todetermine the optimal step-size

min 11988811205821198961 + 11988821205821198962st 1205821 + 1205822 = minus (1 + 21198861 minus (1 + 1198861) ℎ) (1 + 1198861)

12058211205822 = 1198861(1 + 1198861) 0 lt 1205821 lt 1 minus1 lt 1205822 lt 0 0 lt ℎ lt (2 + 41198861)(1 + 1198861)

(48)

However the nonlinear optimization problem (48) is oftendifficult to solve and in the following we give an intuitiveanalyses about the optimal step-size Obviously the smallermax |1205821| |1205822| is the smaller the term 119866119896 is Thus under theconstraint conditions of (48) we aim to minimize the term1205821minus1205822 and equivalently minimize the term (1205821minus1205822)2 whichcan be written as

(1205821 minus 1205822)2 = (1205821 minus 1205822)2 minus 412058211205822= Δ = 1

(1 + 1198861)2 ((1 + 1198861)2 ℎ2

minus 2 (1 + 1198861) (1 + 21198861) ℎ + 1) (49)

Obviously (1205821 minus1205822)2 obtains the minimum value at ℎlowast = (1+21198861)(1 + 1198861) isin (0 (2 + 41198861)(1 + 1198861))(2) If 1198861 isin [0 +infin) Δ is nonnegative (I) For anyℎ isin ((1 + 21198861 + radic1198861(1 + 1198861))(1 + 1198861) (2 + 41198861)(1 + 1198861)) Δ gt 0 Similar to the above analyse the optimal step-size

can be approximately by ℎlowast which belongs to the interval((1 + 21198861 + radic1198861(1 + 1198861))(1 + 1198861) (2 + 41198861)(1 + 1198861)) (II)For any ℎ isin (0 (1 + 21198861 + radic1198861(1 + 1198861))(1 + 1198861)] Δ le 0which indicates that (35) has a multiple root or two complexconjugate roots which are denoted by 1205821 1205822 again and satisfy|1205821| = |1205822| Then according to the general solution formulaof the difference equation we minimize |1205821|2 to approximatethe optimal step-size Obviously |1205821|2 = 1198861(1 + 1198861) which isindependent of the step-size ℎ therefore the optimal step-sizecan also be approximated by ℎlowast

Overall we get the following theorem

Theorem12 For any given 1198861 isin (minus12 +infin) the optimal step-size of the DTZNN model (18) can be approximated by ℎlowast =(1 + 21198861)(1 + 1198861)4 Numerical Results

In this section we present some numerical results to substan-tiate the efficiency and superiority of the proposed DTZNN


=001 =0001

||Ａ(Ｒ

ＥＮＥ)|| 2

102

100

10minus2

10minus4

10minus6

||Ａ(Ｒ

ＥＮＥ)|| 2

101

102

100

10minus1

10minus2

10minus3

10minus4

DTZNNminusIDTZNNminusII


5000 100000k

500 10000k

Figure 2 Trajectories of SSRE 119892(119909119896 119905119896)2 of problem (50) generated by the DTZNNmodels (14) and (18) left 120591 = 001 right 120591 = 0001

model (18) (denoted by DTZNN-I) for TVNO and comparedwith the one-step DTZNNmodel (14) (denoted by DTZNN-II) in [15] All the numerical experiments are performed on anThinkpad laptop with Intel Core 2 CPU 210 GHZ and RAM400 GM All the programs are written in Matlab R2014a

Consider the following TVNO [16]

min119909isinR4

119891 (119909 (119905) 119905) = (1199091 (119905) + 10 sin(12058711990540))2

+ (1199092 (119905) + 1199054)2 + (1199093 (119905) minus exp(minus 1199054))2+ 0025 (119905 minus 1) 1199093 (119905) 1199094 (119905)+ (1199091 (119905) + ln(sin(12058711990540) + 1))2

sdot (1199092 (119905) + sin( 1199058) + cos( 1199058))2minus (1199091 (119905) + (1199091 (119905) + sin (119905))) 1199093 (119905)+ (1199094 (119905) + exp(minus 1199054))2

(50)

andwe can get its stationary point byMatlabwhich is omitteddue to its complicated expressionNowwe useDTZNN-I andDTZNN-II to solve problem (50) and the parameters are setas follows 120591 = 001s or 0001s ℎ = 01 and 1198861 = minus13The initial state vector 1199090 = 119909(0) = [1 2 3 4]⊤ with timeduration being 10s The trajectories of SSRE 119892(119909119896 119905119896)2 ofTVNO problem (50) generated by the two tested DTZNNmodels are depicted in Figure 2

Figure 2 illustrates that the performance of the DTZNNmodel (18) with 1198861 = minus13 is better than that of the DTZNNmodel (14) and both generated SSREs converge to zero in an119874(1205912) manner So when the sampling gap 120591 gt 0 decreasesboth SSREs can be made sufficiently small

Figure 3 depicts the trajectories of the theoretical solution119909lowast119894 (119905119896) (119894 = 1 2 3 4) of problem (50) and 119909119894(119905119896) (119894 = 1 2 3 4)generated by the DTZNN model (18) with 1198861 = minus13 120591 =001 which shows that the numerical results generated bythe DTZNNmodel (18) approximate the theoretical solution119909lowast119894 (119905119896) (119894 = 1 2 3 4) with high accuracy

Figure 4 shows the trajectories of 119891(119909119896 119905119896) generatedby the two tested models and their differences when 120591 =001 from which we can find that 119891(119909119896 119905119896) generated by theDTZNN-I model is generally smaller than that generated bythe DTZNN-II model which means that the former is moreaccurate than the latter

Now let us verify Theorem 11 and we compare thenumerical results generated by the DTZNN model (18) with120591 = 001 1198861 = minus12 ℎ = 01 and those generated by theDTZNN model (18) with 120591 = 001 1198861 = minus13 ℎ = (2 +41198861)(1 + 1198861) = 1 The numerical results are depicted inFigure 5 from which we find that the generated sequence119892(119909119896 119905119896)2 does not converge to zero when 1198861 = minus12 or ℎis equal to the upper bound (2 + 41198861)(1 + 1198861) and these areconsistent to Theorem 11

In the remainder of this section let us verify Theorem 12with 120591 = 001 1198861 = minus13 ℎ = 01 and the optimal step-size ℎ = 05 for fixed 1198861 = minus13 The numerical results aredepicted in Figure 6 which shows the performance of theDTZNNmodel with ℎ = 05 is better than that of theDTZNNmodel with ℎ = 01 and this is consistent to Theorem 12


minus10

0

10

20

30

minus20

minus10

0

10

minus10

0

10

20

30

minus4

minus2

0

2

4

Ｒlowast1 (ＮＥ)

Ｒ1(ＮＥ)

Ｒlowast2 (ＮＥ)

Ｒ2(ＮＥ)

Ｒlowast4 (ＮＥ)

Ｒ4(ＮＥ)

Ｒlowast3 (ＮＥ)

Ｒ3(ＮＥ)

500 10000k

500 10000k

500 10000k

500 10000k

Figure 3 Trajectories of 119909lowast119894 (119905119896) (119894 = 1 2 3 4) and 119909119894(119905119896) (119894 = 1 2 3 4) generated by the DTZNNmodels (14) and (18) and their difference forproblem (50)

20

40

60

80

100

120

140

160

180

200=001=001

102

100

10minus2

10minus4


(ＲＥ

ＮＥ)

500 10000k

500 10000k

））minus）

Figure 4 Trajectories of 119891(119909119897 119905119896) generated by the DTZNNs model (18) for problem (50)


0

1

2

3

4

5

6

7

8

9

10

0

1

2

3

4

5

6

7

8

9

10

||Ａ(Ｒ

ＥＮＥ)|| 2

||Ａ(Ｒ

ＥＮＥ)|| 2

1=minus12h=01 1=minus13h=120

DTZNNminusI

500 10000k

500 10000k

DTZNNminusI

times 10

Figure 5 Trajectories of SSRE 119892(119909119896 119905119896)2 generated by the DTZNNmodel (18) for problem (50)

0 200 400 600 800 1000k

||Ａ(Ｒ

ＥＮＥ)|| 2

=0011=minus13101

100

10minus1

10minus2

10minus3

10minus4

10minus5

h=01h=05

Figure 6 Trajectories of SSRE 119892(119909119896 119905119896)2 generated by the DTZNNmodel (18) with different step-size for problem (50)

5 Conclusion

In this paper we have investigated a convergent two-stepZeaD formulas with truncation error of 119874(1205913) a convergentgeneral two-step ZeaD formula with truncation error of119874(1205912) and a general five-step ZeaD formula with truncationerror of 119874(1205915) which is not convergent Then based on thetwo convergent ZeaD formulas we presented two DTZNNmodels for TVNO and proved that one is divergent and theother with the free parameter 1198861 isin (minus12 +infin) and step-size0 lt ℎ lt (2 + 41198861)(1 + 1198861) is convergent We also proved that

(2+41198861)(1+1198861) is tight upper bound of ℎ and (1+21198861)(1+1198861)is the optimal step-size Numerical results illustrate that theproposed DTZNNmodel is efficient for solving TVNO

In the future the following two issues related to this paperdeserve further studying (I) Theorem 12 only considers theoptimal step-size for any given 1198861 isin (minus12 0) and thereforewe need to study the optimal step-size for any given 1198861 isin(0 +infin) (II) the general three-step DTZNN model and thegeneral four-stepDTZNNmodel proposed in [16 17] both donot give the relationship of the free parameter 1198861 and step-sizeℎ and therefore we are going to extend the technique used in


Theorems 8 and 9 to study the two general multistep DTZNNmodels and explore the relationship of the parameter 1198861 andthe step-size ℎAppendix

A Proof of Lemma 2

Based on (11) the Taylor series expansions of 119891119896+1 and 119891119896minus1 at119909119896 are given as

119891119896+1 = 119891119896 + 120591 119891119896 + 12059122 119891119896 + 119874 (1205913) (A1)

119891119896minus1 = 119891119896 minus 120591 119891119896 + 12059122 119891119896 + 119874 (1205913) (A2)

Substituting (A1) and (A2) into (9) with 119899 = 2 ie119891119896 = 1120591 (1198863119891119896+1 + 1198862119891119896 + 1198861119891119896minus1) + 119874 (120591119901) (A3)

we get the following equation

1198870119891119896 + 1198871120591 119891119896 + 11988721205912 119891119896 + 119874 (1205913) = 119874 (120591119901+1) (A4)

where 1198870 = 1198863 + 1198862 + 1198861 1198871 = 1198863 minus 1198861 minus 1 1198872 = 11988632 + 11988612If 119901 = 2 from Concept 2 to ensure that the two-step ZeaDformula (A3) has a truncation error of 119874(1205913) we only needto ensure that the following conclusions are satisfied

1198870 = 1198863 + 1198862 + 1198861 = 01198871 = 1198863 minus 1198861 minus 1 = 01198872 = 11988632 + 11988612 = 0

(A5)

Solving the above three linear equations we get

1198861 = minus12 1198862 = 01198863 = 12

(A6)

Substituting (A6) into (A3) we get the two-step ZeaDformula (12) with truncation error of 119874(1205913) Then thecharacteristic polynomial of (12) is

120588 (120574) = 1205742 minus 1 = 0 (A7)

whose two roots are minus1 and 1 By Concept 1 the two-stepZeaD formula (12) is zero-stable and thus is convergent byConcept 3

Similarly if 119901 = 1 to ensure that the two-step ZeaDformula (A3) has a truncation error of 119874(1205912) we only needto ensure that the following conclusions are satisfied

1198870 = 1198863 + 1198862 + 1198861 = 01198871 = 1198863 minus 1198861 minus 1 = 0 (A8)

Solving the above two linear equations and letting 1198861 be a freeparameter we get

1198862 = minus1 minus 211988611198863 = 1 + 1198861 (A9)

In this case 1198872 = 1198861 + 12 Then (A4) is reformulated as

(1198861 + 12) 1205912 119891119896 + 119874 (1205913) = 119874 (1205912) (A10)

which is obviously true Substituting (A9) into (A3) weget the two-step ZeaD formula (13) with truncation error of119874(1205912) Then the characteristic polynomial of (13) is

120588 (120574) = (1 + 1198861) 1205742 minus (1 + 21198861) 120574 + 1198861 (A11)

By adopting bilinear transform 120574 = (1 + 1205961205912)(1 minus 1205961205912)[24] we get the following equation

1198882 (1205961205912 )2 + 1198881 (1205961205912 ) + 1198880 = 0 (A12)

where 1198882 = 2+41198861 1198881 = 2 1198880 = 0 Then according the Routhrsquosstability criterion [25] the general two-step ZeaD formula iszero-stable if and only if

1198882 gt 0 ie 1198861 gt minus12 (A13)

Therefore the general two-step ZeaD formula (13) is con-vergent with truncation error of 119874(1205912) if 1198861 gt minus12 Since1198861 = 0 then the effective domain of 1198861 is (minus12 +infin) 0This completes the proof

B Proof of Lemma 5

Based on (11) the Taylor series expansions of 119891119896+1 119891119896minus1 119891119896minus2119891119896minus3 and 119891119896minus4 at 119909119896 are given as

119891119896+1 = 119891119896 + 120591 119891119896 + 12059122 119891119896 + 12059136 119891(3)119896 + 120591424119891(4)119896+ 1205915120119891(5)119896 + 119874 (1205916)

(B1)


119891119896minus1 = 119891119896 minus 120591 119891119896 + 12059122 119891119896 minus 12059136 119891(3)119896 + 120591424119891(4)119896minus 1205915120119891(5)119896 + 119874 (1205916)

(B2)

119891119896minus2 = 119891119896 minus 2120591 119891119896 + 21205912 119891119896 minus 412059133 119891(3)119896 + 212059143 119891(4)119896minus 4120591515 119891(5)119896 + 119874 (1205916)

(B3)


(B4)


(B5)

Substituting (B1)-(B5) into (9) with 119899 = 5 ie

119891119896 = 1120591 (1198866119891119896+1 + 1198865119891119896 + 1198864119891119896minus1 + 1198863119891119896minus2 + 1198862119891119896minus3+ 1198861119891119896minus4) + 119874 (120591119901)

(B6)


1198870119891119896 + 1198871120591 119891119896 + 11988721205912 119891119896 + 11988731205913119891(3)119896 + 11988741205914119891(4)119896+ 11988751205915119891(5)119896 + 119874 (1205916) = 119874 (120591119901+1) (B7)

where

1198870 = 1198866 + 1198865 + 1198864 + 1198863 + 1198862 + 11988611198871 = 1198866 minus 1198864 minus 21198863 minus 31198862 minus 41198861 minus 11198872 = 11988662 + 11988642 + 21198863 + 911988622 + 811988611198873 = 11988666 minus 11988646 minus 411988633 minus 911988622 minus 3211988613 1198874 = 119886624 + 119886424 + 211988633 + 2711988628 + 3211988613 1198875 = 1198866120 minus 1198864120 minus 4119886315 minus 81119886240 minus 128119886115

(B8)

If119901 = 5 fromConcept 2 to ensure that the five-step ZeaDformula (B6) has a truncation error of 119874(1205915) we only needto ensure that the following conclusions are satisfied

1198870 = 1198866 + 1198865 + 1198864 + 1198863 + 1198862 + 1198861 = 01198871 = 1198866 minus 1198864 minus 21198863 minus 31198862 minus 41198861 minus 1 = 01198872 = 11988662 + 11988642 + 21198863 + 911988622 + 81198861 = 01198873 = 11988666 minus 11988646 minus 411988633 minus 911988622 minus 3211988613 = 01198874 = 119886624 + 119886424 + 211988633 + 2711988628 + 3211988613 = 01198875 = 1198866120 minus 1198864120 minus 4119886315 minus 81119886240 minus 128119886115 = 0

(B9)

Solving the above six linear equations with respect to 119886119894 (119894 =1 2 6) we get1198861 = 120 1198862 = minus13 1198863 = 11198864 = minus21198865 = 1312 1198866 = 15

(B10)

Substituting (B10) into (B6) we get a five-step ZeaD formulawith truncation error of 119874(1205916) whose characteristic polyno-mial is

120588 (120574) = 121205745 + 651205744 minus 1201205743 + 601205742 minus 20120574 + 3 = 0 (B11)of which the roots are 1 minus69614 026698 013887 - 033945119894and 013887 + 033945119894 with 119894 denoting imaginary unit ByConcept 1 the resulting five-step ZeaD formula is not zero-stable since the root minus69614 lies outside unit disk Thereforethe five-step ZeaD formula with truncation error of 119874(1205916) isnot zero-stable and thus is not convergent

Similarly if 119901 = 4 to ensure that the five-step ZeaDformula (B6) has a truncation error of119874(1205915) we only need toensure that the first five linear equations of (B9) holds thatis the following conclusions are satisfied

1198870 = 1198866 + 1198865 + 1198864 + 1198863 + 1198862 + 1198861 = 01198871 = 1198866 minus 1198864 minus 21198863 minus 31198862 minus 41198861 minus 1 = 01198872 = 11988662 + 11988642 + 21198863 + 911988622 + 81198861 = 01198873 = 11988666 minus 11988646 minus 411988633 minus 911988622 minus 3211988613 = 01198874 = 119886624 + 119886424 + 211988633 + 2711988628 + 3211988613 = 0

(B12)


Solving the above five linear equations and letting 1198866 be a freeparameter we have

1198861 = 14 minus 11988661198862 = minus34 + 511988661198863 = 3 minus 1011988661198864 = minus4 + 1011988661198865 = 2512 minus 51198866

(B13)

In this case 1198875 = minus15 + 1198866 Then (B7) is reformulated as

(minus15 + 1198866) 1205915119891(5)119896 + 119874 (1205916) = 119874 (1205915) (B14)

which is obviously true Substituting (B13) into (B6) we canget a general five-step ZeaD formula with truncation error of119874(1205915) whose characteristic polynomial is

120588 (120574) = 11988661205745 + (2512 minus 51198866) 1205744 + (101198866 minus 4) 1205743+ (3 minus 101198866) 1205742 + (51198866 minus 43) 120574 minus 1198866 + 14

(B15)

Then by adopting bilinear transform 120574 = (1 + 1205961205912)(1 minus1205961205912) again we get the following equation1198885 (1205961205912 )5 + 1198884 (1205961205912 )4 + 1198883 (1205961205912 )3 + 1198882 (1205961205912 )2

+ 1198881 (1205961205912 ) + 1198880 = 0(B16)

where 1198885 = 321198866 minus 323 1198884 = minus2 1198883 = 143 1198882 = 6 1198881 = 2 1198880 =0 Unfortunately 1198884 = minus2 lt 0 Then according the Routhrsquosstability criterion again the general five-step ZeaD formulawith truncation error of 119874(1205915) is not zero-stable and thus isdivergent This completes the proof

Data Availability

The data used to support the findings of this study areavailable from the corresponding author upon request

Conflicts of Interest

The authors declare that there are no conflicts of interestregarding the publication of this article

Acknowledgments

This research was partially supported by the National NaturalScience Foundation of China and Shandong Province (Nos11671228 11601475 and ZR2016AL05)

References

[1] W XiongOperations ResearchMachinery Industry Press 2014[2] M Sun Y Wang and J Liu ldquoGeneralized Peaceman-Rachford

splitting method for multiple-block separable convex program-ming with applications to robust PCArdquo Calcolo vol 54 no 1pp 77ndash94 2017

[3] Y J Wang and N H Xiu Nonlinear Programming eory andAlgorithm Shaanxi Science and Technology Press 2008

[4] M Sun and J Liu ldquoA modified HestenesndashStiefel projectionmethod for constrained nonlinear equations and its linearconvergence raterdquo Applied Mathematics and Computation vol49 no 1-2 pp 145ndash156 2015

[5] M Sun and J Liu ldquoNew hybrid conjugate gradient projectionmethod for the convex constrained equationsrdquo Calcolo vol 53no 3 pp 399ndash411 2016

[6] M Sun and Q Bai ldquoA new descent memory gradient methodand its global convergencerdquo Journal of Systems Science ampComplexity vol 24 no 4 pp 784ndash794 2011

[7] J Nocedal ldquoUpdating quasi-newton matrices with limitedstoragerdquoMathematics of Computation vol 35 no 151 pp 773ndash782 1980

[8] J L Wang and H NWu ldquoDiscrete-time Zhang neural networkof O(1205913) pattern for time-varying matrix pseudoinversion withapplication to manipulator motion generationrdquo Neurocomput-ing vol 142 pp 165ndash173 2014

[9] W Bian L Ma S Qin and X Xue ldquoNeural network fornonsmooth pseudoconvex optimization with general convexconstraintsrdquo Neural Networks vol 101 pp 1ndash14 2018

[10] A Hosseini J Wang and S M Hosseini ldquoA recurrent neuralnetwork for solving a class of generalized convex optimizationproblemsrdquo Neural Networks vol 44 pp 78ndash86 2013

[11] Y Xia and J Wang ldquoA bi-projection neural network forsolving constrained quadratic optimization problemsrdquo IEEETransactions on Neural Networks and Learning Systems vol 27no 2 pp 214ndash224 2016

[12] DGuo X Lin Z Su S Sun andZHuang ldquoDesign and analysisof two discrete-time ZD algorithms for time-varying nonlinearminimizationrdquo Numerical Algorithms vol 77 no 1 pp 23ndash362018

[13] Y N Zhang Z Li C F Yi and K Chen ldquoZhang neuralnetwork versus gradient neural network for online time-varyingquadratic function minimizationrdquo ICIC vol LNAI 5227 pp807ndash814 2008

[14] Y Zhang M Yang J Li L He and S Wu ldquoZFD formula 4I gSFD Y applied to future minimizationrdquo Physics Letters A vol381 no 19 pp 1677ndash1681 2017

[15] Long Jin and Yunong Zhang ldquoDiscrete-time Zhang neuralnetwork for online time-varying nonlinear optimization withapplication to manipulator motion generationrdquo IEEE Transac-tions on Neural Networks and Learning Systems vol 26 no 7pp 1525ndash1531 2015

[16] Y Zhang L He C Hu J Guo J Li and Y Shi ldquoGeneralfour-step discrete-time zeroing andderivative dynamics appliedto time-varying nonlinear optimizationrdquo Journal of Computa-tional and Applied Mathematics vol 347 pp 314ndash329 2019

[17] CHu X Kang andY Zhang ldquoThree-step general discrete-timeZhang neural network design and application to time-variantmatrix inversionrdquo Neurocomputing vol 306 pp 108ndash118 2018

[18] Y N Zhang and C Yi Zhang Neural Networks and Neural-Dynamic Method Nova New York NY USA 2011


[19] Y Zhang L Jin D Guo Y Yin and Y Chou ldquoTaylor-type 1-step-ahead numerical differentiation rule for first-orderderivative approximation and ZNN discretizationrdquo Journal ofComputational and Applied Mathematics vol 273 pp 29ndash402015

[20] D F Griffiths and D J Higham Numerical Methods for Ordi-nary Differential Equations Initial Value Problems SpringerLondon UK 2010

[21] Y N Zhang M J Zhu C W Hu J Li and M Yang ldquoEuler-precision general-form of Zhang et al discretization (ZeaD) for-mulas derivation and numerical experimentsrdquo in Proceedingsof Chinese Control and Decision Conference (CCDC) pp 6273ndash6278 2018

[22] Y N Zhang J J Guo L He Y Shi and C W Hu ldquoAny ZeaDformula of six instants having no quartic or higher precisionwith proofrdquo in Proceedings of the 5th International Conferenceon Systems and Informatics (ICSAI) pp 681ndash685 IEEENanjingChina 2018

[23] G H Golub X Wu and J-Y Yuan ldquoSOR-like methods foraugmented systemsrdquo BIT Numerical Mathematics vol 41 no1 pp 71ndash85 2001

[24] A V Oppenheim Discrete-Time Signal Processing PearsonHigher Education Inc New Jersey NJ USA 2010

[25] K Ogata Modern Control Engineering Prentice-Hall Engle-wood Cliffs NJ USA 2001

Hindawiwwwhindawicom Volume 2018

MathematicsJournal of


Mathematical Problems in Engineering

Applied MathematicsJournal of


Probability and StatisticsHindawiwwwhindawicom Volume 2018

Journal of


Mathematical PhysicsAdvances in

Complex AnalysisJournal of


OptimizationJournal of



Engineering Mathematics

International Journal of


Operations ResearchAdvances in

Journal of


Function SpacesAbstract and Applied AnalysisHindawiwwwhindawicom Volume 2018

International Journal of Mathematics and Mathematical Sciences


Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom

The Scientific World Journal

Volume 2018

Hindawiwwwhindawicom Volume 2018Volume 2018

Numerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisAdvances inAdvances in Discrete Dynamics in

Nature and SocietyHindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom

Dierential EquationsInternational Journal of

Volume 2018


Decision SciencesAdvances in


AnalysisInternational Journal of


Stochastic AnalysisInternational Journal of

Submit your manuscripts atwwwhindawicom


Table 1 ZeaD formulas used in [12 15ndash17]

Formula 119899-step General form Truncation error

[15] 119891119896 = 119891119896+1 minus 119891119896120591 1 No 119874(1205912)[12] 119891119896 = 2119891119896+1 minus 3119891119896 + 2119891119896minus1 minus 119891119896minus22120591 3 No 119874(1205913)[12] 119891119896 = 6119891119896+1 minus 3119891119896 minus 2119891119896minus1 minus 119891119896minus210120591 3 No 119874(1205913)[17] 119891119896 = (minus1198861 + 12)119891119896+1 + 31198861119891119896 minus (31198861 + 12)119891119896minus1 + 1198861119891119896minus2120591 3 Yes 119874(1205913)[16] 119891119896 = (1198861 + 13)119891119896+1 minus (41198861 minus 12)119891119896 + (61198861 minus 1)119891119896minus1120591

minus (41198861 minus 16)119891119896minus2 minus 1198861119891119896minus3120591 4 Yes 119874(1205914)

Most of the above algorithms are designed intrinsicallyfor solving static nonlinear optimization (SNO) thereforethey might not be effective enough for solving time-varyingnonlinear optimization (TVNO) whose objective functiondenoted by 119891(119909(119905) 119905) is a multivariate function with respectto the decision variable 119909 and the time variable 119905 Theconnection between SNO and TVNO is obvious (1) SNO isa special case of TVNO and when 119909(119905) = 119909 for any 119905 ge 0TVNO reduces to SNO (2) the discrete-time form of TVNOwhose objective function is 119891(119909119896 119905119896) (119905119896 = 119896120591 119896 = 0 1 )and 120591 gt 0 denoting the sampling gap can be viewed asa sequence of SNO At each single time instant 119905 = 119905119896TVNO can be viewed as a static nonlinear optimizationand consequently it can be solved by the above mentionedalgorithms However this treatment is not advisable due tothe following three reasons (1) it is usually inefficient andlowly precise due to it need to solve a sequence of SNO [12](2) in the online solution process of discrete-time TVNOthe present andor previous data with respect to 119909119894 (119894 le 119896)should be used sufficiently to generate the unknown decisionvariable 119909119896+1 (3) most importantly at time instant 119905119896 we donot know the future information such as the function values119891(119909(119905119896+1) 119905119896+1) and only the current and past informationfor time instances 119905119895 with 119895 le 119896 can be used Therefore theconventional static methods and the neural networks in [9ndash11] which are based on the future information cannot solveTVNO

As a special recurrent neural network Zhang neuralnetwork (ZNN) named after Chinese scholar ZhangYunongserves as a unified approach to solve various online time-varying problems such as time-varying quadratic functionminimization [13] future minimization [14] time-varyingmatrix pseudoinversion [8] and TVNO [12 15 16] Forexample based on ZNN Jin et al [15] presented a one-stepdiscrete-time ZNN (DTZNN)model for TVNO whosemax-imal residual error is theoretically O(1205912) Subsequently Guoet al [12] proposed two DTZNN models for TVNO whichbelong to three-step DTZNN with steady-state residual error(SSRE) changing in an O(1205913) manner Then quite recently

Zhang et al [16] presented a general four-step discrete-timederivative dynamics model and a general four-step DTZNNmodel for TVNO both models contain a free parameterwhich can take any values of the interval (112 16) and isconvergent with truncation error of O(1205914)

Generally speaking in the real-time solution processof TVNO the more the past data is utilized the smallerthe truncation error of the corresponding Zhang et aldiscretization (ZeaD) formula is For example the truncationerrors of the one-step DTZNN model in [15] the three-step DTZNN models in [12] the general three-step DTZNNmodels in [17] and the four-step DTZNN models in [16]are O(1205912) O(1205913) O(1205913) and O(1205914) respectively The corre-sponding ZeaD formulas are summarized in Table 1 in whichthe effective domains of the parameter 1198861 in [17] and [16] are(minusinfin 0) and (112 16) respectively Obviously the generalZeaD formula with 1198861 = minus12 minus110 in [17] reduces the twospecial ZeaD formulas in [12]

In this paper we are going to further study the ZeaDformula and three ZeaD formulas are presented including aspecial two-step ZeaD formula with truncation error 119874(1205913)a general two-step ZeaD formula with truncation error119874(1205912) and a general five-step ZeaD formula with truncationerror 119874(1205915) We prove that the first two ZeaD formulasis convergent while the third ZeaD formula is not zero-stable and thus is divergent Then based on the Taylor seriesexpansion and the above two convergent ZeaD formulaswe discrete the continuous-time ZNN (CTZNN) model forTVNO and thus get a special two-step DTZNN model anda general two-step DTZNN model for TVNO Theoreticalanalyses indicate that the first DTZNN model is divergentwhile the second DTZNN model is convergent for any 1198861 isin(minus12 +infin) and the step-size ℎ isin (0 (2 + 41198861)(1 + 1198861)) Inaddition the tight upper bound of the step-size ℎ and theoptimal step-size are also discussed

The rest of the paper is organized as follows We firstrecall some basic definitions and results in Section 2 includ-ing problem formulation of TVNO continuous-time ZNN(CTZNN) model for TVNO and the general 119899-step ZeaD







2 Preliminaries



min119909(119905)isinR119899

119891 (119909 (119905) 119905) isin R 119905 isin [0 119905119891] (1)







119890 (119905) ≐ 119889119890 (119905)119889119905 = minus120574119890 (119905) 120574 gt 0 (3)


(119905) = minus119867 (119909 (119905) 119905)minus1 (120574119892 (119909 (119905) 119905) + 119892119905 (119909 (119905) 119905)) (4)


= [ 12059721198911205971199091120597119905 12059721198911205971199092120597119905

1205972119891120597119909119899120597119905]⊤ isin R

119899(5)


119867(119909 (119905) 119905) =

[[[[[[[[[[[[[[[


d

1205972119891120597119909119899120597119909112059721198911205971199091198991205971199092 sdot sdot sdot 1205972119891120597119909119899120597119909119899

]]]]]]]]]]]]]]]


(6)



(119905) = 119865 (119909) (7)


(119905) = minus120574119867 (119909)minus1 119892 (119909) (8)




119891119896 = 1120591 (119899+1sum119894=1

119886119894119891119896minus119899+119894) + 119874 (120591119901) (9)






119909119896+1 + 119899sum119894=1

120572119894119909119896+1minus119894 = 120591 119899sum119894=0

120573119894V119896+1minus119894 (10)



(ie not multiple)





119891119896+1 = 119891119896 + 120591 119891119896 + 12059122 119891119896 + 12059133119891119896 + + 120591119898119898119891(119898)119896

+ 119874 (120591119898+1) (11)



119891119896 = 119891119896+1 minus 119891119896minus12120591 (12)


119891119896 = (1 + 1198861) 119891119896+1 minus (1 + 21198861) 119891119896 + 1198861119891119896minus1120591 (13)




119909119896+1 = 119909119896 minus 119867 (119909119896 119905119896)minus1 (ℎ119892 (119909119896 119905119896) + 120591119892119905 (119909119896 119905119896)) (14)


(1198861 + 12) 1205912 119891119896 (15)






119876 (1198866) = max 1003816100381610038161003816120574119894 (1198866)1003816100381610038161003816 | 119894 = 1 2 5 (16)






119909119896+1= 1 + 211988611 + 1198861 119909119896 minus


(18)




119909119896+1 = 1 + 211988611 + 1198861 119909119896 minus11988611 + 1198861 119909119896minus1

minus ℎ119867 (119909119896 119905119896)minus1 119892 (119909119896 119905119896) (20)



119909119896+1 = 119866 (119909119896 119905119896) (21)

minus100 minus50 0 50 1000

5

10

15

20

25

Q

a6

Q=1Q=Q(a6)




(22)


119892 (119909119896+1 119905119896+1) = 119892 (119909119896 119905119896) + 119867 (119909119896 119905119896) Δ119909119896+ 120591119892119905 (119909119896 119905119896) + O (1205912) (23)


and




+ O (1205912) (25)



(26)

ie



119866119896+1 + ℎ119866119896 minus 119866119896minus1 = 0 (28)


1205822 + ℎ120582 minus 1 = 0 (29)





(1 + 1198861) 119892 (119909119896+1 119905119896+1) + 1198861119892 (119909119896minus1 119905119896minus1)= (1 + 21198861) 119892 (119909119896 119905119896)

+ 119867 (119909119896 119905119896) ((1 + 1198861) Δ119909119896 minus 1198861Δ119909119896minus1)+ 120591119892119905 (119909119896 119905119896) + O (1205912)

(31)



ie


= 1 + 21198861 minus (1 + 1198861) ℎ1 + 1198861 (119892 (119909119896 119905119896) minus O (1205912)) )(33)




1205822 minus 1 + 21198861 minus (1 + 1198861) ℎ1 + 1198861 120582 + 11988611 + 1198861 = 0 (35)


1003816100381610038161003816100381610038161003816100381611988611 + 1198861

10038161003816100381610038161003816100381610038161003816 lt 11003816100381610038161003816100381610038161003816100381610038161 + 21198861 minus (1 + 1198861) ℎ1 + 1198861

100381610038161003816100381610038161003816100381610038161003816 lt 1 + 11988611 + 1198861 (36)



So

0 lt ℎ lt 2 + 411988611 + 1198861 (39)




1199091 = 1199090 minus 119867 (1199090 1199050)minus1 (ℎ119892 (1199090 1199050) + 120591119892119905 (1199090 1199050)) (40)




1205822 + 1 + 211988611 + 1198861 120582 + 11988611 + 1198861 = 0 (41)


(42)


119866119896 = 1198881 (minus1)119896 + 1198882 ( 11988611 + 1198861)119896 (43)



Δ = 1(1 + 1198861)2 (ℎ

2 minus 2 (1 + 21198861) ℎ + 1) (44)


Δ = 41198861 (1 + 1198861) (45)



1205821 + 1205822 = minus1 + 21198861 minus (1 + 1198861) ℎ1 + 1198861 12058211205822 = 11988611 + 1198861

(46)





(48)



(1 + 1198861)2 ((1 + 1198861)2 ℎ2

minus 2 (1 + 1198861) (1 + 21198861) ℎ + 1) (49)







=001 =0001

||Ａ(Ｒ

ＥＮＥ)|| 2

102

100

10minus2

10minus4

10minus6

||Ａ(Ｒ

ＥＮＥ)|| 2

101

102

100

10minus1

10minus2

10minus3

10minus4



5000 100000k

500 10000k




min119909isinR4

119891 (119909 (119905) 119905) = (1199091 (119905) + 10 sin(12058711990540))2



(50)








minus10

0

10

20

30

minus20

minus10

0

10

minus10

0

10

20

30

minus4

minus2

0

2

4

Ｒlowast1 (ＮＥ)

Ｒ1(ＮＥ)

Ｒlowast2 (ＮＥ)

Ｒ2(ＮＥ)

Ｒlowast4 (ＮＥ)

Ｒ4(ＮＥ)

Ｒlowast3 (ＮＥ)

Ｒ3(ＮＥ)

500 10000k

500 10000k

500 10000k

500 10000k


20

40

60

80

100

120

140

160

180

200=001=001

102

100

10minus2

10minus4


(ＲＥ

ＮＥ)

500 10000k

500 10000k

））minus）



0

1

2

3

4

5

6

7

8

9

10

0

1

2

3

4

5

6

7

8

9

10

||Ａ(Ｒ

ＥＮＥ)|| 2

||Ａ(Ｒ

ＥＮＥ)|| 2


DTZNNminusI

500 10000k

500 10000k

DTZNNminusI

times 10


0 200 400 600 800 1000k

||Ａ(Ｒ

ＥＮＥ)|| 2

=0011=minus13101

100

10minus1

10minus2

10minus3

10minus4

10minus5

h=01h=05


5 Conclusion






A Proof of Lemma 2


119891119896+1 = 119891119896 + 120591 119891119896 + 12059122 119891119896 + 119874 (1205913) (A1)

119891119896minus1 = 119891119896 minus 120591 119891119896 + 12059122 119891119896 + 119874 (1205913) (A2)



1198870119891119896 + 1198871120591 119891119896 + 11988721205912 119891119896 + 119874 (1205913) = 119874 (120591119901+1) (A4)


1198870 = 1198863 + 1198862 + 1198861 = 01198871 = 1198863 minus 1198861 minus 1 = 01198872 = 11988632 + 11988612 = 0

(A5)


1198861 = minus12 1198862 = 01198863 = 12

(A6)


120588 (120574) = 1205742 minus 1 = 0 (A7)



1198870 = 1198863 + 1198862 + 1198861 = 01198871 = 1198863 minus 1198861 minus 1 = 0 (A8)


1198862 = minus1 minus 211988611198863 = 1 + 1198861 (A9)


(1198861 + 12) 1205912 119891119896 + 119874 (1205913) = 119874 (1205912) (A10)


120588 (120574) = (1 + 1198861) 1205742 minus (1 + 21198861) 120574 + 1198861 (A11)


1198882 (1205961205912 )2 + 1198881 (1205961205912 ) + 1198880 = 0 (A12)


1198882 gt 0 ie 1198861 gt minus12 (A13)


B Proof of Lemma 5


119891119896+1 = 119891119896 + 120591 119891119896 + 12059122 119891119896 + 12059136 119891(3)119896 + 120591424119891(4)119896+ 1205915120119891(5)119896 + 119874 (1205916)

(B1)



(B2)


(B3)


(B4)


(B5)



(B6)


1198870119891119896 + 1198871120591 119891119896 + 11988721205912 119891119896 + 11988731205913119891(3)119896 + 11988741205914119891(4)119896+ 11988751205915119891(5)119896 + 119874 (1205916) = 119874 (120591119901+1) (B7)

where


(B8)



(B9)


(B10)





(B12)




(B13)


(minus15 + 1198866) 1205915119891(5)119896 + 119874 (1205916) = 119874 (1205915) (B14)



(B15)


+ 1198881 (1205961205912 ) + 1198880 = 0(B16)


Data Availability




Acknowledgments


References


































Journal of












Journal of







Volume 2018






Volume 2018














2 Preliminaries



min119909(119905)isinR119899

119891 (119909 (119905) 119905) isin R 119905 isin [0 119905119891] (1)







119890 (119905) ≐ 119889119890 (119905)119889119905 = minus120574119890 (119905) 120574 gt 0 (3)


(119905) = minus119867 (119909 (119905) 119905)minus1 (120574119892 (119909 (119905) 119905) + 119892119905 (119909 (119905) 119905)) (4)


= [ 12059721198911205971199091120597119905 12059721198911205971199092120597119905

1205972119891120597119909119899120597119905]⊤ isin R

119899(5)


119867(119909 (119905) 119905) =

[[[[[[[[[[[[[[[


d

1205972119891120597119909119899120597119909112059721198911205971199091198991205971199092 sdot sdot sdot 1205972119891120597119909119899120597119909119899

]]]]]]]]]]]]]]]


(6)



(119905) = 119865 (119909) (7)


(119905) = minus120574119867 (119909)minus1 119892 (119909) (8)




119891119896 = 1120591 (119899+1sum119894=1

119886119894119891119896minus119899+119894) + 119874 (120591119901) (9)






119909119896+1 + 119899sum119894=1

120572119894119909119896+1minus119894 = 120591 119899sum119894=0

120573119894V119896+1minus119894 (10)



(ie not multiple)





119891119896+1 = 119891119896 + 120591 119891119896 + 12059122 119891119896 + 12059133119891119896 + + 120591119898119898119891(119898)119896

+ 119874 (120591119898+1) (11)



119891119896 = 119891119896+1 minus 119891119896minus12120591 (12)


119891119896 = (1 + 1198861) 119891119896+1 minus (1 + 21198861) 119891119896 + 1198861119891119896minus1120591 (13)




119909119896+1 = 119909119896 minus 119867 (119909119896 119905119896)minus1 (ℎ119892 (119909119896 119905119896) + 120591119892119905 (119909119896 119905119896)) (14)


(1198861 + 12) 1205912 119891119896 (15)






119876 (1198866) = max 1003816100381610038161003816120574119894 (1198866)1003816100381610038161003816 | 119894 = 1 2 5 (16)






119909119896+1= 1 + 211988611 + 1198861 119909119896 minus


(18)




119909119896+1 = 1 + 211988611 + 1198861 119909119896 minus11988611 + 1198861 119909119896minus1

minus ℎ119867 (119909119896 119905119896)minus1 119892 (119909119896 119905119896) (20)



119909119896+1 = 119866 (119909119896 119905119896) (21)

minus100 minus50 0 50 1000

5

10

15

20

25

Q

a6

Q=1Q=Q(a6)




(22)


119892 (119909119896+1 119905119896+1) = 119892 (119909119896 119905119896) + 119867 (119909119896 119905119896) Δ119909119896+ 120591119892119905 (119909119896 119905119896) + O (1205912) (23)


and




+ O (1205912) (25)



(26)

ie



119866119896+1 + ℎ119866119896 minus 119866119896minus1 = 0 (28)


1205822 + ℎ120582 minus 1 = 0 (29)





(1 + 1198861) 119892 (119909119896+1 119905119896+1) + 1198861119892 (119909119896minus1 119905119896minus1)= (1 + 21198861) 119892 (119909119896 119905119896)

+ 119867 (119909119896 119905119896) ((1 + 1198861) Δ119909119896 minus 1198861Δ119909119896minus1)+ 120591119892119905 (119909119896 119905119896) + O (1205912)

(31)



ie


= 1 + 21198861 minus (1 + 1198861) ℎ1 + 1198861 (119892 (119909119896 119905119896) minus O (1205912)) )(33)




1205822 minus 1 + 21198861 minus (1 + 1198861) ℎ1 + 1198861 120582 + 11988611 + 1198861 = 0 (35)


1003816100381610038161003816100381610038161003816100381611988611 + 1198861

10038161003816100381610038161003816100381610038161003816 lt 11003816100381610038161003816100381610038161003816100381610038161 + 21198861 minus (1 + 1198861) ℎ1 + 1198861

100381610038161003816100381610038161003816100381610038161003816 lt 1 + 11988611 + 1198861 (36)



So

0 lt ℎ lt 2 + 411988611 + 1198861 (39)




1199091 = 1199090 minus 119867 (1199090 1199050)minus1 (ℎ119892 (1199090 1199050) + 120591119892119905 (1199090 1199050)) (40)




1205822 + 1 + 211988611 + 1198861 120582 + 11988611 + 1198861 = 0 (41)


(42)


119866119896 = 1198881 (minus1)119896 + 1198882 ( 11988611 + 1198861)119896 (43)



Δ = 1(1 + 1198861)2 (ℎ

2 minus 2 (1 + 21198861) ℎ + 1) (44)


Δ = 41198861 (1 + 1198861) (45)



1205821 + 1205822 = minus1 + 21198861 minus (1 + 1198861) ℎ1 + 1198861 12058211205822 = 11988611 + 1198861

(46)





(48)



(1 + 1198861)2 ((1 + 1198861)2 ℎ2

minus 2 (1 + 1198861) (1 + 21198861) ℎ + 1) (49)







=001 =0001

||Ａ(Ｒ

ＥＮＥ)|| 2

102

100

10minus2

10minus4

10minus6

||Ａ(Ｒ

ＥＮＥ)|| 2

101

102

100

10minus1

10minus2

10minus3

10minus4



5000 100000k

500 10000k




min119909isinR4

119891 (119909 (119905) 119905) = (1199091 (119905) + 10 sin(12058711990540))2



(50)








minus10

0

10

20

30

minus20

minus10

0

10

minus10

0

10

20

30

minus4

minus2

0

2

4

Ｒlowast1 (ＮＥ)

Ｒ1(ＮＥ)

Ｒlowast2 (ＮＥ)

Ｒ2(ＮＥ)

Ｒlowast4 (ＮＥ)

Ｒ4(ＮＥ)

Ｒlowast3 (ＮＥ)

Ｒ3(ＮＥ)

500 10000k

500 10000k

500 10000k

500 10000k


20

40

60

80

100

120

140

160

180

200=001=001

102

100

10minus2

10minus4


(ＲＥ

ＮＥ)

500 10000k

500 10000k

））minus）



0

1

2

3

4

5

6

7

8

9

10

0

1

2

3

4

5

6

7

8

9

10

||Ａ(Ｒ

ＥＮＥ)|| 2

||Ａ(Ｒ

ＥＮＥ)|| 2


DTZNNminusI

500 10000k

500 10000k

DTZNNminusI

times 10


0 200 400 600 800 1000k

||Ａ(Ｒ

ＥＮＥ)|| 2

=0011=minus13101

100

10minus1

10minus2

10minus3

10minus4

10minus5

h=01h=05


5 Conclusion






A Proof of Lemma 2


119891119896+1 = 119891119896 + 120591 119891119896 + 12059122 119891119896 + 119874 (1205913) (A1)

119891119896minus1 = 119891119896 minus 120591 119891119896 + 12059122 119891119896 + 119874 (1205913) (A2)



1198870119891119896 + 1198871120591 119891119896 + 11988721205912 119891119896 + 119874 (1205913) = 119874 (120591119901+1) (A4)


1198870 = 1198863 + 1198862 + 1198861 = 01198871 = 1198863 minus 1198861 minus 1 = 01198872 = 11988632 + 11988612 = 0

(A5)


1198861 = minus12 1198862 = 01198863 = 12

(A6)


120588 (120574) = 1205742 minus 1 = 0 (A7)



1198870 = 1198863 + 1198862 + 1198861 = 01198871 = 1198863 minus 1198861 minus 1 = 0 (A8)


1198862 = minus1 minus 211988611198863 = 1 + 1198861 (A9)


(1198861 + 12) 1205912 119891119896 + 119874 (1205913) = 119874 (1205912) (A10)


120588 (120574) = (1 + 1198861) 1205742 minus (1 + 21198861) 120574 + 1198861 (A11)


1198882 (1205961205912 )2 + 1198881 (1205961205912 ) + 1198880 = 0 (A12)


1198882 gt 0 ie 1198861 gt minus12 (A13)


B Proof of Lemma 5


119891119896+1 = 119891119896 + 120591 119891119896 + 12059122 119891119896 + 12059136 119891(3)119896 + 120591424119891(4)119896+ 1205915120119891(5)119896 + 119874 (1205916)

(B1)



(B2)


(B3)


(B4)


(B5)



(B6)


1198870119891119896 + 1198871120591 119891119896 + 11988721205912 119891119896 + 11988731205913119891(3)119896 + 11988741205914119891(4)119896+ 11988751205915119891(5)119896 + 119874 (1205916) = 119874 (120591119901+1) (B7)

where


(B8)



(B9)


(B10)





(B12)




(B13)


(minus15 + 1198866) 1205915119891(5)119896 + 119874 (1205916) = 119874 (1205915) (B14)



(B15)


+ 1198881 (1205961205912 ) + 1198880 = 0(B16)


Data Availability




Acknowledgments


References


































Journal of












Journal of







Volume 2018






Volume 2018











119891119896 = 1120591 (119899+1sum119894=1

119886119894119891119896minus119899+119894) + 119874 (120591119901) (9)






119909119896+1 + 119899sum119894=1

120572119894119909119896+1minus119894 = 120591 119899sum119894=0

120573119894V119896+1minus119894 (10)



(ie not multiple)





119891119896+1 = 119891119896 + 120591 119891119896 + 12059122 119891119896 + 12059133119891119896 + + 120591119898119898119891(119898)119896

+ 119874 (120591119898+1) (11)



119891119896 = 119891119896+1 minus 119891119896minus12120591 (12)


119891119896 = (1 + 1198861) 119891119896+1 minus (1 + 21198861) 119891119896 + 1198861119891119896minus1120591 (13)




119909119896+1 = 119909119896 minus 119867 (119909119896 119905119896)minus1 (ℎ119892 (119909119896 119905119896) + 120591119892119905 (119909119896 119905119896)) (14)


(1198861 + 12) 1205912 119891119896 (15)






119876 (1198866) = max 1003816100381610038161003816120574119894 (1198866)1003816100381610038161003816 | 119894 = 1 2 5 (16)






119909119896+1= 1 + 211988611 + 1198861 119909119896 minus


(18)




119909119896+1 = 1 + 211988611 + 1198861 119909119896 minus11988611 + 1198861 119909119896minus1

minus ℎ119867 (119909119896 119905119896)minus1 119892 (119909119896 119905119896) (20)



119909119896+1 = 119866 (119909119896 119905119896) (21)

minus100 minus50 0 50 1000

5

10

15

20

25

Q

a6

Q=1Q=Q(a6)




(22)


119892 (119909119896+1 119905119896+1) = 119892 (119909119896 119905119896) + 119867 (119909119896 119905119896) Δ119909119896+ 120591119892119905 (119909119896 119905119896) + O (1205912) (23)


and




+ O (1205912) (25)



(26)

ie



119866119896+1 + ℎ119866119896 minus 119866119896minus1 = 0 (28)


1205822 + ℎ120582 minus 1 = 0 (29)





(1 + 1198861) 119892 (119909119896+1 119905119896+1) + 1198861119892 (119909119896minus1 119905119896minus1)= (1 + 21198861) 119892 (119909119896 119905119896)

+ 119867 (119909119896 119905119896) ((1 + 1198861) Δ119909119896 minus 1198861Δ119909119896minus1)+ 120591119892119905 (119909119896 119905119896) + O (1205912)

(31)



ie


= 1 + 21198861 minus (1 + 1198861) ℎ1 + 1198861 (119892 (119909119896 119905119896) minus O (1205912)) )(33)




1205822 minus 1 + 21198861 minus (1 + 1198861) ℎ1 + 1198861 120582 + 11988611 + 1198861 = 0 (35)


1003816100381610038161003816100381610038161003816100381611988611 + 1198861

10038161003816100381610038161003816100381610038161003816 lt 11003816100381610038161003816100381610038161003816100381610038161 + 21198861 minus (1 + 1198861) ℎ1 + 1198861

100381610038161003816100381610038161003816100381610038161003816 lt 1 + 11988611 + 1198861 (36)



So

0 lt ℎ lt 2 + 411988611 + 1198861 (39)




1199091 = 1199090 minus 119867 (1199090 1199050)minus1 (ℎ119892 (1199090 1199050) + 120591119892119905 (1199090 1199050)) (40)




1205822 + 1 + 211988611 + 1198861 120582 + 11988611 + 1198861 = 0 (41)


(42)


119866119896 = 1198881 (minus1)119896 + 1198882 ( 11988611 + 1198861)119896 (43)



Δ = 1(1 + 1198861)2 (ℎ

2 minus 2 (1 + 21198861) ℎ + 1) (44)


Δ = 41198861 (1 + 1198861) (45)



1205821 + 1205822 = minus1 + 21198861 minus (1 + 1198861) ℎ1 + 1198861 12058211205822 = 11988611 + 1198861

(46)





(48)



(1 + 1198861)2 ((1 + 1198861)2 ℎ2

minus 2 (1 + 1198861) (1 + 21198861) ℎ + 1) (49)







=001 =0001

||Ａ(Ｒ

ＥＮＥ)|| 2

102

100

10minus2

10minus4

10minus6

||Ａ(Ｒ

ＥＮＥ)|| 2

101

102

100

10minus1

10minus2

10minus3

10minus4



5000 100000k

500 10000k




min119909isinR4

119891 (119909 (119905) 119905) = (1199091 (119905) + 10 sin(12058711990540))2



(50)








minus10

0

10

20

30

minus20

minus10

0

10

minus10

0

10

20

30

minus4

minus2

0

2

4

Ｒlowast1 (ＮＥ)

Ｒ1(ＮＥ)

Ｒlowast2 (ＮＥ)

Ｒ2(ＮＥ)

Ｒlowast4 (ＮＥ)

Ｒ4(ＮＥ)

Ｒlowast3 (ＮＥ)

Ｒ3(ＮＥ)

500 10000k

500 10000k

500 10000k

500 10000k


20

40

60

80

100

120

140

160

180

200=001=001

102

100

10minus2

10minus4


(ＲＥ

ＮＥ)

500 10000k

500 10000k

））minus）



0

1

2

3

4

5

6

7

8

9

10

0

1

2

3

4

5

6

7

8

9

10

||Ａ(Ｒ

ＥＮＥ)|| 2

||Ａ(Ｒ

ＥＮＥ)|| 2


DTZNNminusI

500 10000k

500 10000k

DTZNNminusI

times 10


0 200 400 600 800 1000k

||Ａ(Ｒ

ＥＮＥ)|| 2

=0011=minus13101

100

10minus1

10minus2

10minus3

10minus4

10minus5

h=01h=05


5 Conclusion






A Proof of Lemma 2


119891119896+1 = 119891119896 + 120591 119891119896 + 12059122 119891119896 + 119874 (1205913) (A1)

119891119896minus1 = 119891119896 minus 120591 119891119896 + 12059122 119891119896 + 119874 (1205913) (A2)



1198870119891119896 + 1198871120591 119891119896 + 11988721205912 119891119896 + 119874 (1205913) = 119874 (120591119901+1) (A4)


1198870 = 1198863 + 1198862 + 1198861 = 01198871 = 1198863 minus 1198861 minus 1 = 01198872 = 11988632 + 11988612 = 0

(A5)


1198861 = minus12 1198862 = 01198863 = 12

(A6)


120588 (120574) = 1205742 minus 1 = 0 (A7)



1198870 = 1198863 + 1198862 + 1198861 = 01198871 = 1198863 minus 1198861 minus 1 = 0 (A8)


1198862 = minus1 minus 211988611198863 = 1 + 1198861 (A9)


(1198861 + 12) 1205912 119891119896 + 119874 (1205913) = 119874 (1205912) (A10)


120588 (120574) = (1 + 1198861) 1205742 minus (1 + 21198861) 120574 + 1198861 (A11)


1198882 (1205961205912 )2 + 1198881 (1205961205912 ) + 1198880 = 0 (A12)


1198882 gt 0 ie 1198861 gt minus12 (A13)


B Proof of Lemma 5


119891119896+1 = 119891119896 + 120591 119891119896 + 12059122 119891119896 + 12059136 119891(3)119896 + 120591424119891(4)119896+ 1205915120119891(5)119896 + 119874 (1205916)

(B1)



(B2)


(B3)


(B4)


(B5)



(B6)


1198870119891119896 + 1198871120591 119891119896 + 11988721205912 119891119896 + 11988731205913119891(3)119896 + 11988741205914119891(4)119896+ 11988751205915119891(5)119896 + 119874 (1205916) = 119874 (120591119901+1) (B7)

where


(B8)



(B9)


(B10)





(B12)




(B13)


(minus15 + 1198866) 1205915119891(5)119896 + 119874 (1205916) = 119874 (1205915) (B14)



(B15)


+ 1198881 (1205961205912 ) + 1198880 = 0(B16)


Data Availability




Acknowledgments


References


































Journal of












Journal of







Volume 2018






Volume 2018










119876 (1198866) = max 1003816100381610038161003816120574119894 (1198866)1003816100381610038161003816 | 119894 = 1 2 5 (16)






119909119896+1= 1 + 211988611 + 1198861 119909119896 minus


(18)




119909119896+1 = 1 + 211988611 + 1198861 119909119896 minus11988611 + 1198861 119909119896minus1

minus ℎ119867 (119909119896 119905119896)minus1 119892 (119909119896 119905119896) (20)



119909119896+1 = 119866 (119909119896 119905119896) (21)

minus100 minus50 0 50 1000

5

10

15

20

25

Q

a6

Q=1Q=Q(a6)




(22)


119892 (119909119896+1 119905119896+1) = 119892 (119909119896 119905119896) + 119867 (119909119896 119905119896) Δ119909119896+ 120591119892119905 (119909119896 119905119896) + O (1205912) (23)


and




+ O (1205912) (25)



(26)

ie



119866119896+1 + ℎ119866119896 minus 119866119896minus1 = 0 (28)


1205822 + ℎ120582 minus 1 = 0 (29)





(1 + 1198861) 119892 (119909119896+1 119905119896+1) + 1198861119892 (119909119896minus1 119905119896minus1)= (1 + 21198861) 119892 (119909119896 119905119896)

+ 119867 (119909119896 119905119896) ((1 + 1198861) Δ119909119896 minus 1198861Δ119909119896minus1)+ 120591119892119905 (119909119896 119905119896) + O (1205912)

(31)



ie


= 1 + 21198861 minus (1 + 1198861) ℎ1 + 1198861 (119892 (119909119896 119905119896) minus O (1205912)) )(33)




1205822 minus 1 + 21198861 minus (1 + 1198861) ℎ1 + 1198861 120582 + 11988611 + 1198861 = 0 (35)


1003816100381610038161003816100381610038161003816100381611988611 + 1198861

10038161003816100381610038161003816100381610038161003816 lt 11003816100381610038161003816100381610038161003816100381610038161 + 21198861 minus (1 + 1198861) ℎ1 + 1198861

100381610038161003816100381610038161003816100381610038161003816 lt 1 + 11988611 + 1198861 (36)



So

0 lt ℎ lt 2 + 411988611 + 1198861 (39)




1199091 = 1199090 minus 119867 (1199090 1199050)minus1 (ℎ119892 (1199090 1199050) + 120591119892119905 (1199090 1199050)) (40)




1205822 + 1 + 211988611 + 1198861 120582 + 11988611 + 1198861 = 0 (41)


(42)


119866119896 = 1198881 (minus1)119896 + 1198882 ( 11988611 + 1198861)119896 (43)



Δ = 1(1 + 1198861)2 (ℎ

2 minus 2 (1 + 21198861) ℎ + 1) (44)


Δ = 41198861 (1 + 1198861) (45)



1205821 + 1205822 = minus1 + 21198861 minus (1 + 1198861) ℎ1 + 1198861 12058211205822 = 11988611 + 1198861

(46)





(48)



(1 + 1198861)2 ((1 + 1198861)2 ℎ2

minus 2 (1 + 1198861) (1 + 21198861) ℎ + 1) (49)







=001 =0001

||Ａ(Ｒ

ＥＮＥ)|| 2

102

100

10minus2

10minus4

10minus6

||Ａ(Ｒ

ＥＮＥ)|| 2

101

102

100

10minus1

10minus2

10minus3

10minus4



5000 100000k

500 10000k




min119909isinR4

119891 (119909 (119905) 119905) = (1199091 (119905) + 10 sin(12058711990540))2



(50)








minus10

0

10

20

30

minus20

minus10

0

10

minus10

0

10

20

30

minus4

minus2

0

2

4

Ｒlowast1 (ＮＥ)

Ｒ1(ＮＥ)

Ｒlowast2 (ＮＥ)

Ｒ2(ＮＥ)

Ｒlowast4 (ＮＥ)

Ｒ4(ＮＥ)

Ｒlowast3 (ＮＥ)

Ｒ3(ＮＥ)

500 10000k

500 10000k

500 10000k

500 10000k


20

40

60

80

100

120

140

160

180

200=001=001

102

100

10minus2

10minus4


(ＲＥ

ＮＥ)

500 10000k

500 10000k

））minus）



0

1

2

3

4

5

6

7

8

9

10

0

1

2

3

4

5

6

7

8

9

10

||Ａ(Ｒ

ＥＮＥ)|| 2

||Ａ(Ｒ

ＥＮＥ)|| 2


DTZNNminusI

500 10000k

500 10000k

DTZNNminusI

times 10


0 200 400 600 800 1000k

||Ａ(Ｒ

ＥＮＥ)|| 2

=0011=minus13101

100

10minus1

10minus2

10minus3

10minus4

10minus5

h=01h=05


5 Conclusion






A Proof of Lemma 2


119891119896+1 = 119891119896 + 120591 119891119896 + 12059122 119891119896 + 119874 (1205913) (A1)

119891119896minus1 = 119891119896 minus 120591 119891119896 + 12059122 119891119896 + 119874 (1205913) (A2)



1198870119891119896 + 1198871120591 119891119896 + 11988721205912 119891119896 + 119874 (1205913) = 119874 (120591119901+1) (A4)


1198870 = 1198863 + 1198862 + 1198861 = 01198871 = 1198863 minus 1198861 minus 1 = 01198872 = 11988632 + 11988612 = 0

(A5)


1198861 = minus12 1198862 = 01198863 = 12

(A6)


120588 (120574) = 1205742 minus 1 = 0 (A7)



1198870 = 1198863 + 1198862 + 1198861 = 01198871 = 1198863 minus 1198861 minus 1 = 0 (A8)


1198862 = minus1 minus 211988611198863 = 1 + 1198861 (A9)


(1198861 + 12) 1205912 119891119896 + 119874 (1205913) = 119874 (1205912) (A10)


120588 (120574) = (1 + 1198861) 1205742 minus (1 + 21198861) 120574 + 1198861 (A11)


1198882 (1205961205912 )2 + 1198881 (1205961205912 ) + 1198880 = 0 (A12)


1198882 gt 0 ie 1198861 gt minus12 (A13)


B Proof of Lemma 5


119891119896+1 = 119891119896 + 120591 119891119896 + 12059122 119891119896 + 12059136 119891(3)119896 + 120591424119891(4)119896+ 1205915120119891(5)119896 + 119874 (1205916)

(B1)



(B2)


(B3)


(B4)


(B5)



(B6)


1198870119891119896 + 1198871120591 119891119896 + 11988721205912 119891119896 + 11988731205913119891(3)119896 + 11988741205914119891(4)119896+ 11988751205915119891(5)119896 + 119874 (1205916) = 119874 (120591119901+1) (B7)

where


(B8)



(B9)


(B10)





(B12)




(B13)


(minus15 + 1198866) 1205915119891(5)119896 + 119874 (1205916) = 119874 (1205915) (B14)



(B15)


+ 1198881 (1205961205912 ) + 1198880 = 0(B16)


Data Availability




Acknowledgments


References


































Journal of












Journal of







Volume 2018






Volume 2018









and




+ O (1205912) (25)



(26)

ie



119866119896+1 + ℎ119866119896 minus 119866119896minus1 = 0 (28)


1205822 + ℎ120582 minus 1 = 0 (29)





(1 + 1198861) 119892 (119909119896+1 119905119896+1) + 1198861119892 (119909119896minus1 119905119896minus1)= (1 + 21198861) 119892 (119909119896 119905119896)

+ 119867 (119909119896 119905119896) ((1 + 1198861) Δ119909119896 minus 1198861Δ119909119896minus1)+ 120591119892119905 (119909119896 119905119896) + O (1205912)

(31)



ie


= 1 + 21198861 minus (1 + 1198861) ℎ1 + 1198861 (119892 (119909119896 119905119896) minus O (1205912)) )(33)




1205822 minus 1 + 21198861 minus (1 + 1198861) ℎ1 + 1198861 120582 + 11988611 + 1198861 = 0 (35)


1003816100381610038161003816100381610038161003816100381611988611 + 1198861

10038161003816100381610038161003816100381610038161003816 lt 11003816100381610038161003816100381610038161003816100381610038161 + 21198861 minus (1 + 1198861) ℎ1 + 1198861

100381610038161003816100381610038161003816100381610038161003816 lt 1 + 11988611 + 1198861 (36)



So

0 lt ℎ lt 2 + 411988611 + 1198861 (39)




1199091 = 1199090 minus 119867 (1199090 1199050)minus1 (ℎ119892 (1199090 1199050) + 120591119892119905 (1199090 1199050)) (40)




1205822 + 1 + 211988611 + 1198861 120582 + 11988611 + 1198861 = 0 (41)


(42)


119866119896 = 1198881 (minus1)119896 + 1198882 ( 11988611 + 1198861)119896 (43)



Δ = 1(1 + 1198861)2 (ℎ

2 minus 2 (1 + 21198861) ℎ + 1) (44)


Δ = 41198861 (1 + 1198861) (45)



1205821 + 1205822 = minus1 + 21198861 minus (1 + 1198861) ℎ1 + 1198861 12058211205822 = 11988611 + 1198861

(46)





(48)



(1 + 1198861)2 ((1 + 1198861)2 ℎ2

minus 2 (1 + 1198861) (1 + 21198861) ℎ + 1) (49)







=001 =0001

||Ａ(Ｒ

ＥＮＥ)|| 2

102

100

10minus2

10minus4

10minus6

||Ａ(Ｒ

ＥＮＥ)|| 2

101

102

100

10minus1

10minus2

10minus3

10minus4



5000 100000k

500 10000k




min119909isinR4

119891 (119909 (119905) 119905) = (1199091 (119905) + 10 sin(12058711990540))2



(50)








minus10

0

10

20

30

minus20

minus10

0

10

minus10

0

10

20

30

minus4

minus2

0

2

4

Ｒlowast1 (ＮＥ)

Ｒ1(ＮＥ)

Ｒlowast2 (ＮＥ)

Ｒ2(ＮＥ)

Ｒlowast4 (ＮＥ)

Ｒ4(ＮＥ)

Ｒlowast3 (ＮＥ)

Ｒ3(ＮＥ)

500 10000k

500 10000k

500 10000k

500 10000k


20

40

60

80

100

120

140

160

180

200=001=001

102

100

10minus2

10minus4


(ＲＥ

ＮＥ)

500 10000k

500 10000k

））minus）



0

1

2

3

4

5

6

7

8

9

10

0

1

2

3

4

5

6

7

8

9

10

||Ａ(Ｒ

ＥＮＥ)|| 2

||Ａ(Ｒ

ＥＮＥ)|| 2


DTZNNminusI

500 10000k

500 10000k

DTZNNminusI

times 10


0 200 400 600 800 1000k

||Ａ(Ｒ

ＥＮＥ)|| 2

=0011=minus13101

100

10minus1

10minus2

10minus3

10minus4

10minus5

h=01h=05


5 Conclusion






A Proof of Lemma 2


119891119896+1 = 119891119896 + 120591 119891119896 + 12059122 119891119896 + 119874 (1205913) (A1)

119891119896minus1 = 119891119896 minus 120591 119891119896 + 12059122 119891119896 + 119874 (1205913) (A2)



1198870119891119896 + 1198871120591 119891119896 + 11988721205912 119891119896 + 119874 (1205913) = 119874 (120591119901+1) (A4)


1198870 = 1198863 + 1198862 + 1198861 = 01198871 = 1198863 minus 1198861 minus 1 = 01198872 = 11988632 + 11988612 = 0

(A5)


1198861 = minus12 1198862 = 01198863 = 12

(A6)


120588 (120574) = 1205742 minus 1 = 0 (A7)



1198870 = 1198863 + 1198862 + 1198861 = 01198871 = 1198863 minus 1198861 minus 1 = 0 (A8)


1198862 = minus1 minus 211988611198863 = 1 + 1198861 (A9)


(1198861 + 12) 1205912 119891119896 + 119874 (1205913) = 119874 (1205912) (A10)


120588 (120574) = (1 + 1198861) 1205742 minus (1 + 21198861) 120574 + 1198861 (A11)


1198882 (1205961205912 )2 + 1198881 (1205961205912 ) + 1198880 = 0 (A12)


1198882 gt 0 ie 1198861 gt minus12 (A13)


B Proof of Lemma 5


119891119896+1 = 119891119896 + 120591 119891119896 + 12059122 119891119896 + 12059136 119891(3)119896 + 120591424119891(4)119896+ 1205915120119891(5)119896 + 119874 (1205916)

(B1)



(B2)


(B3)


(B4)


(B5)



(B6)


1198870119891119896 + 1198871120591 119891119896 + 11988721205912 119891119896 + 11988731205913119891(3)119896 + 11988741205914119891(4)119896+ 11988751205915119891(5)119896 + 119874 (1205916) = 119874 (120591119901+1) (B7)

where


(B8)



(B9)


(B10)





(B12)




(B13)


(minus15 + 1198866) 1205915119891(5)119896 + 119874 (1205916) = 119874 (1205915) (B14)



(B15)


+ 1198881 (1205961205912 ) + 1198880 = 0(B16)


Data Availability




Acknowledgments


References


































Journal of












Journal of







Volume 2018






Volume 2018










1199091 = 1199090 minus 119867 (1199090 1199050)minus1 (ℎ119892 (1199090 1199050) + 120591119892119905 (1199090 1199050)) (40)




1205822 + 1 + 211988611 + 1198861 120582 + 11988611 + 1198861 = 0 (41)


(42)


119866119896 = 1198881 (minus1)119896 + 1198882 ( 11988611 + 1198861)119896 (43)



Δ = 1(1 + 1198861)2 (ℎ

2 minus 2 (1 + 21198861) ℎ + 1) (44)


Δ = 41198861 (1 + 1198861) (45)



1205821 + 1205822 = minus1 + 21198861 minus (1 + 1198861) ℎ1 + 1198861 12058211205822 = 11988611 + 1198861

(46)





(48)



(1 + 1198861)2 ((1 + 1198861)2 ℎ2

minus 2 (1 + 1198861) (1 + 21198861) ℎ + 1) (49)







=001 =0001

||Ａ(Ｒ

ＥＮＥ)|| 2

102

100

10minus2

10minus4

10minus6

||Ａ(Ｒ

ＥＮＥ)|| 2

101

102

100

10minus1

10minus2

10minus3

10minus4



5000 100000k

500 10000k




min119909isinR4

119891 (119909 (119905) 119905) = (1199091 (119905) + 10 sin(12058711990540))2



(50)








minus10

0

10

20

30

minus20

minus10

0

10

minus10

0

10

20

30

minus4

minus2

0

2

4

Ｒlowast1 (ＮＥ)

Ｒ1(ＮＥ)

Ｒlowast2 (ＮＥ)

Ｒ2(ＮＥ)

Ｒlowast4 (ＮＥ)

Ｒ4(ＮＥ)

Ｒlowast3 (ＮＥ)

Ｒ3(ＮＥ)

500 10000k

500 10000k

500 10000k

500 10000k


20

40

60

80

100

120

140

160

180

200=001=001

102

100

10minus2

10minus4


(ＲＥ

ＮＥ)

500 10000k

500 10000k

））minus）



0

1

2

3

4

5

6

7

8

9

10

0

1

2

3

4

5

6

7

8

9

10

||Ａ(Ｒ

ＥＮＥ)|| 2

||Ａ(Ｒ

ＥＮＥ)|| 2


DTZNNminusI

500 10000k

500 10000k

DTZNNminusI

times 10


0 200 400 600 800 1000k

||Ａ(Ｒ

ＥＮＥ)|| 2

=0011=minus13101

100

10minus1

10minus2

10minus3

10minus4

10minus5

h=01h=05


5 Conclusion






A Proof of Lemma 2


119891119896+1 = 119891119896 + 120591 119891119896 + 12059122 119891119896 + 119874 (1205913) (A1)

119891119896minus1 = 119891119896 minus 120591 119891119896 + 12059122 119891119896 + 119874 (1205913) (A2)



1198870119891119896 + 1198871120591 119891119896 + 11988721205912 119891119896 + 119874 (1205913) = 119874 (120591119901+1) (A4)


1198870 = 1198863 + 1198862 + 1198861 = 01198871 = 1198863 minus 1198861 minus 1 = 01198872 = 11988632 + 11988612 = 0

(A5)


1198861 = minus12 1198862 = 01198863 = 12

(A6)


120588 (120574) = 1205742 minus 1 = 0 (A7)



1198870 = 1198863 + 1198862 + 1198861 = 01198871 = 1198863 minus 1198861 minus 1 = 0 (A8)


1198862 = minus1 minus 211988611198863 = 1 + 1198861 (A9)


(1198861 + 12) 1205912 119891119896 + 119874 (1205913) = 119874 (1205912) (A10)


120588 (120574) = (1 + 1198861) 1205742 minus (1 + 21198861) 120574 + 1198861 (A11)


1198882 (1205961205912 )2 + 1198881 (1205961205912 ) + 1198880 = 0 (A12)


1198882 gt 0 ie 1198861 gt minus12 (A13)


B Proof of Lemma 5


119891119896+1 = 119891119896 + 120591 119891119896 + 12059122 119891119896 + 12059136 119891(3)119896 + 120591424119891(4)119896+ 1205915120119891(5)119896 + 119874 (1205916)

(B1)



(B2)


(B3)


(B4)


(B5)



(B6)


1198870119891119896 + 1198871120591 119891119896 + 11988721205912 119891119896 + 11988731205913119891(3)119896 + 11988741205914119891(4)119896+ 11988751205915119891(5)119896 + 119874 (1205916) = 119874 (120591119901+1) (B7)

where


(B8)



(B9)


(B10)





(B12)




(B13)


(minus15 + 1198866) 1205915119891(5)119896 + 119874 (1205916) = 119874 (1205915) (B14)



(B15)


+ 1198881 (1205961205912 ) + 1198880 = 0(B16)


Data Availability




Acknowledgments


References


































Journal of












Journal of







Volume 2018






Volume 2018









=001 =0001

||Ａ(Ｒ

ＥＮＥ)|| 2

102

100

10minus2

10minus4

10minus6

||Ａ(Ｒ

ＥＮＥ)|| 2

101

102

100

10minus1

10minus2

10minus3

10minus4



5000 100000k

500 10000k




min119909isinR4

119891 (119909 (119905) 119905) = (1199091 (119905) + 10 sin(12058711990540))2



(50)








minus10

0

10

20

30

minus20

minus10

0

10

minus10

0

10

20

30

minus4

minus2

0

2

4

Ｒlowast1 (ＮＥ)

Ｒ1(ＮＥ)

Ｒlowast2 (ＮＥ)

Ｒ2(ＮＥ)

Ｒlowast4 (ＮＥ)

Ｒ4(ＮＥ)

Ｒlowast3 (ＮＥ)

Ｒ3(ＮＥ)

500 10000k

500 10000k

500 10000k

500 10000k


20

40

60

80

100

120

140

160

180

200=001=001

102

100

10minus2

10minus4


(ＲＥ

ＮＥ)

500 10000k

500 10000k

））minus）



0

1

2

3

4

5

6

7

8

9

10

0

1

2

3

4

5

6

7

8

9

10

||Ａ(Ｒ

ＥＮＥ)|| 2

||Ａ(Ｒ

ＥＮＥ)|| 2


DTZNNminusI

500 10000k

500 10000k

DTZNNminusI

times 10


0 200 400 600 800 1000k

||Ａ(Ｒ

ＥＮＥ)|| 2

=0011=minus13101

100

10minus1

10minus2

10minus3

10minus4

10minus5

h=01h=05


5 Conclusion






A Proof of Lemma 2


119891119896+1 = 119891119896 + 120591 119891119896 + 12059122 119891119896 + 119874 (1205913) (A1)

119891119896minus1 = 119891119896 minus 120591 119891119896 + 12059122 119891119896 + 119874 (1205913) (A2)



1198870119891119896 + 1198871120591 119891119896 + 11988721205912 119891119896 + 119874 (1205913) = 119874 (120591119901+1) (A4)


1198870 = 1198863 + 1198862 + 1198861 = 01198871 = 1198863 minus 1198861 minus 1 = 01198872 = 11988632 + 11988612 = 0

(A5)


1198861 = minus12 1198862 = 01198863 = 12

(A6)


120588 (120574) = 1205742 minus 1 = 0 (A7)



1198870 = 1198863 + 1198862 + 1198861 = 01198871 = 1198863 minus 1198861 minus 1 = 0 (A8)


1198862 = minus1 minus 211988611198863 = 1 + 1198861 (A9)


(1198861 + 12) 1205912 119891119896 + 119874 (1205913) = 119874 (1205912) (A10)


120588 (120574) = (1 + 1198861) 1205742 minus (1 + 21198861) 120574 + 1198861 (A11)


1198882 (1205961205912 )2 + 1198881 (1205961205912 ) + 1198880 = 0 (A12)


1198882 gt 0 ie 1198861 gt minus12 (A13)


B Proof of Lemma 5


119891119896+1 = 119891119896 + 120591 119891119896 + 12059122 119891119896 + 12059136 119891(3)119896 + 120591424119891(4)119896+ 1205915120119891(5)119896 + 119874 (1205916)

(B1)



(B2)


(B3)


(B4)


(B5)



(B6)


1198870119891119896 + 1198871120591 119891119896 + 11988721205912 119891119896 + 11988731205913119891(3)119896 + 11988741205914119891(4)119896+ 11988751205915119891(5)119896 + 119874 (1205916) = 119874 (120591119901+1) (B7)

where


(B8)



(B9)


(B10)





(B12)




(B13)


(minus15 + 1198866) 1205915119891(5)119896 + 119874 (1205916) = 119874 (1205915) (B14)



(B15)


+ 1198881 (1205961205912 ) + 1198880 = 0(B16)


Data Availability




Acknowledgments


References


































Journal of












Journal of







Volume 2018






Volume 2018









minus10

0

10

20

30

minus20

minus10

0

10

minus10

0

10

20

30

minus4

minus2

0

2

4

Ｒlowast1 (ＮＥ)

Ｒ1(ＮＥ)

Ｒlowast2 (ＮＥ)

Ｒ2(ＮＥ)

Ｒlowast4 (ＮＥ)

Ｒ4(ＮＥ)

Ｒlowast3 (ＮＥ)

Ｒ3(ＮＥ)

500 10000k

500 10000k

500 10000k

500 10000k


20

40

60

80

100

120

140

160

180

200=001=001

102

100

10minus2

10minus4


(ＲＥ

ＮＥ)

500 10000k

500 10000k

））minus）



0

1

2

3

4

5

6

7

8

9

10

0

1

2

3

4

5

6

7

8

9

10

||Ａ(Ｒ

ＥＮＥ)|| 2

||Ａ(Ｒ

ＥＮＥ)|| 2


DTZNNminusI

500 10000k

500 10000k

DTZNNminusI

times 10


0 200 400 600 800 1000k

||Ａ(Ｒ

ＥＮＥ)|| 2

=0011=minus13101

100

10minus1

10minus2

10minus3

10minus4

10minus5

h=01h=05


5 Conclusion






A Proof of Lemma 2


119891119896+1 = 119891119896 + 120591 119891119896 + 12059122 119891119896 + 119874 (1205913) (A1)

119891119896minus1 = 119891119896 minus 120591 119891119896 + 12059122 119891119896 + 119874 (1205913) (A2)



1198870119891119896 + 1198871120591 119891119896 + 11988721205912 119891119896 + 119874 (1205913) = 119874 (120591119901+1) (A4)


1198870 = 1198863 + 1198862 + 1198861 = 01198871 = 1198863 minus 1198861 minus 1 = 01198872 = 11988632 + 11988612 = 0

(A5)


1198861 = minus12 1198862 = 01198863 = 12

(A6)


120588 (120574) = 1205742 minus 1 = 0 (A7)



1198870 = 1198863 + 1198862 + 1198861 = 01198871 = 1198863 minus 1198861 minus 1 = 0 (A8)


1198862 = minus1 minus 211988611198863 = 1 + 1198861 (A9)


(1198861 + 12) 1205912 119891119896 + 119874 (1205913) = 119874 (1205912) (A10)


120588 (120574) = (1 + 1198861) 1205742 minus (1 + 21198861) 120574 + 1198861 (A11)


1198882 (1205961205912 )2 + 1198881 (1205961205912 ) + 1198880 = 0 (A12)


1198882 gt 0 ie 1198861 gt minus12 (A13)


B Proof of Lemma 5


119891119896+1 = 119891119896 + 120591 119891119896 + 12059122 119891119896 + 12059136 119891(3)119896 + 120591424119891(4)119896+ 1205915120119891(5)119896 + 119874 (1205916)

(B1)



(B2)


(B3)


(B4)


(B5)



(B6)


1198870119891119896 + 1198871120591 119891119896 + 11988721205912 119891119896 + 11988731205913119891(3)119896 + 11988741205914119891(4)119896+ 11988751205915119891(5)119896 + 119874 (1205916) = 119874 (120591119901+1) (B7)

where


(B8)



(B9)


(B10)





(B12)




(B13)


(minus15 + 1198866) 1205915119891(5)119896 + 119874 (1205916) = 119874 (1205915) (B14)



(B15)


+ 1198881 (1205961205912 ) + 1198880 = 0(B16)


Data Availability




Acknowledgments


References


































Journal of












Journal of







Volume 2018






Volume 2018









0

1

2

3

4

5

6

7

8

9

10

0

1

2

3

4

5

6

7

8

9

10

||Ａ(Ｒ

ＥＮＥ)|| 2

||Ａ(Ｒ

ＥＮＥ)|| 2


DTZNNminusI

500 10000k

500 10000k

DTZNNminusI

times 10


0 200 400 600 800 1000k

||Ａ(Ｒ

ＥＮＥ)|| 2

=0011=minus13101

100

10minus1

10minus2

10minus3

10minus4

10minus5

h=01h=05


5 Conclusion






A Proof of Lemma 2


119891119896+1 = 119891119896 + 120591 119891119896 + 12059122 119891119896 + 119874 (1205913) (A1)

119891119896minus1 = 119891119896 minus 120591 119891119896 + 12059122 119891119896 + 119874 (1205913) (A2)



1198870119891119896 + 1198871120591 119891119896 + 11988721205912 119891119896 + 119874 (1205913) = 119874 (120591119901+1) (A4)


1198870 = 1198863 + 1198862 + 1198861 = 01198871 = 1198863 minus 1198861 minus 1 = 01198872 = 11988632 + 11988612 = 0

(A5)


1198861 = minus12 1198862 = 01198863 = 12

(A6)


120588 (120574) = 1205742 minus 1 = 0 (A7)



1198870 = 1198863 + 1198862 + 1198861 = 01198871 = 1198863 minus 1198861 minus 1 = 0 (A8)


1198862 = minus1 minus 211988611198863 = 1 + 1198861 (A9)


(1198861 + 12) 1205912 119891119896 + 119874 (1205913) = 119874 (1205912) (A10)


120588 (120574) = (1 + 1198861) 1205742 minus (1 + 21198861) 120574 + 1198861 (A11)


1198882 (1205961205912 )2 + 1198881 (1205961205912 ) + 1198880 = 0 (A12)


1198882 gt 0 ie 1198861 gt minus12 (A13)


B Proof of Lemma 5


119891119896+1 = 119891119896 + 120591 119891119896 + 12059122 119891119896 + 12059136 119891(3)119896 + 120591424119891(4)119896+ 1205915120119891(5)119896 + 119874 (1205916)

(B1)



(B2)


(B3)


(B4)


(B5)



(B6)


1198870119891119896 + 1198871120591 119891119896 + 11988721205912 119891119896 + 11988731205913119891(3)119896 + 11988741205914119891(4)119896+ 11988751205915119891(5)119896 + 119874 (1205916) = 119874 (120591119901+1) (B7)

where


(B8)



(B9)


(B10)





(B12)




(B13)


(minus15 + 1198866) 1205915119891(5)119896 + 119874 (1205916) = 119874 (1205915) (B14)



(B15)


+ 1198881 (1205961205912 ) + 1198880 = 0(B16)


Data Availability




Acknowledgments


References


































Journal of












Journal of







Volume 2018






Volume 2018










A Proof of Lemma 2


119891119896+1 = 119891119896 + 120591 119891119896 + 12059122 119891119896 + 119874 (1205913) (A1)

119891119896minus1 = 119891119896 minus 120591 119891119896 + 12059122 119891119896 + 119874 (1205913) (A2)



1198870119891119896 + 1198871120591 119891119896 + 11988721205912 119891119896 + 119874 (1205913) = 119874 (120591119901+1) (A4)


1198870 = 1198863 + 1198862 + 1198861 = 01198871 = 1198863 minus 1198861 minus 1 = 01198872 = 11988632 + 11988612 = 0

(A5)


1198861 = minus12 1198862 = 01198863 = 12

(A6)


120588 (120574) = 1205742 minus 1 = 0 (A7)



1198870 = 1198863 + 1198862 + 1198861 = 01198871 = 1198863 minus 1198861 minus 1 = 0 (A8)


1198862 = minus1 minus 211988611198863 = 1 + 1198861 (A9)


(1198861 + 12) 1205912 119891119896 + 119874 (1205913) = 119874 (1205912) (A10)


120588 (120574) = (1 + 1198861) 1205742 minus (1 + 21198861) 120574 + 1198861 (A11)


1198882 (1205961205912 )2 + 1198881 (1205961205912 ) + 1198880 = 0 (A12)


1198882 gt 0 ie 1198861 gt minus12 (A13)


B Proof of Lemma 5


119891119896+1 = 119891119896 + 120591 119891119896 + 12059122 119891119896 + 12059136 119891(3)119896 + 120591424119891(4)119896+ 1205915120119891(5)119896 + 119874 (1205916)

(B1)



(B2)


(B3)


(B4)


(B5)



(B6)


1198870119891119896 + 1198871120591 119891119896 + 11988721205912 119891119896 + 11988731205913119891(3)119896 + 11988741205914119891(4)119896+ 11988751205915119891(5)119896 + 119874 (1205916) = 119874 (120591119901+1) (B7)

where


(B8)



(B9)


(B10)





(B12)




(B13)


(minus15 + 1198866) 1205915119891(5)119896 + 119874 (1205916) = 119874 (1205915) (B14)



(B15)


+ 1198881 (1205961205912 ) + 1198880 = 0(B16)


Data Availability




Acknowledgments


References


































Journal of












Journal of







Volume 2018






Volume 2018










(B2)


(B3)


(B4)


(B5)



(B6)


1198870119891119896 + 1198871120591 119891119896 + 11988721205912 119891119896 + 11988731205913119891(3)119896 + 11988741205914119891(4)119896+ 11988751205915119891(5)119896 + 119874 (1205916) = 119874 (120591119901+1) (B7)

where


(B8)



(B9)


(B10)





(B12)




(B13)


(minus15 + 1198866) 1205915119891(5)119896 + 119874 (1205916) = 119874 (1205915) (B14)



(B15)


+ 1198881 (1205961205912 ) + 1198880 = 0(B16)


Data Availability




Acknowledgments


References


































Journal of












Journal of







Volume 2018






Volume 2018











(B13)


(minus15 + 1198866) 1205915119891(5)119896 + 119874 (1205916) = 119874 (1205915) (B14)



(B15)


+ 1198881 (1205961205912 ) + 1198880 = 0(B16)


Data Availability




Acknowledgments


References


































Journal of












Journal of







Volume 2018






Volume 2018























Journal of












Journal of







Volume 2018






Volume 2018








discrete-time zhang neural networks for time-varying nonlinear optimizationdownloads.hindawi.com ›...

Documents