mathematics of operations research · mathematics of operations research 38(4), pp. 740–760,...

This article was downloaded by: [158.132.175.28] On: 15 September 2015, At: 05:12Publisher: Institute for Operations Research and the Management Sciences (INFORMS)INFORMS is located in Maryland, USA

Mathematics of Operations Research

Publication details, including instructions for authors and subscription information:http://pubsonline.informs.org

Nonlinear Augmented Lagrangian and Duality TheoryC. Y. Wang, X. Q. Yang, X. M. Yang

To cite this article:C. Y. Wang, X. Q. Yang, X. M. Yang (2013) Nonlinear Augmented Lagrangian and Duality Theory. Mathematics of OperationsResearch 38(4):740-760. http://dx.doi.org/10.1287/moor.2013.0591

Full terms and conditions of use: http://pubsonline.informs.org/page/terms-and-conditions

This article may be used only for the purposes of research, teaching, and/or private study. Commercial useor systematic downloading (by robots or other automatic processes) is prohibited without explicit Publisherapproval, unless otherwise noted. For more information, contact [email protected].

The Publisher does not warrant or guarantee the article’s accuracy, completeness, merchantability, fitnessfor a particular purpose, or non-infringement. Descriptions of, or references to, products or publications, orinclusion of an advertisement in this article, neither constitutes nor implies a guarantee, endorsement, orsupport of claims made of that product, publication, or service.

Copyright © 2013, INFORMS

Please scroll down for article—it is on subsequent pages

INFORMS is the largest professional society in the world for professionals in the fields of operations research, managementscience, and analytics.For more information on INFORMS, its publications, membership, or meetings visit http://www.informs.org

http://pubsonline.informs.org

http://dx.doi.org/10.1287/moor.2013.0591

http://pubsonline.informs.org/page/terms-and-conditions

http://www.informs.org

MATHEMATICS OF OPERATIONS RESEARCH

Vol. 38, No. 4, November 2013, pp. 740–760ISSN 0364-765X (print) � ISSN 1526-5471 (online)

http://dx.doi.org/10.1287/moor.2013.0591© 2013 INFORMS

Nonlinear Augmented Lagrangian and Duality Theory

C. Y. WangInstitute of Operations Research, Qufu Normal University, Qufu, China, [email protected]

X. Q. YangDepartment of Applied Mathematics, The Hong Kong Polytechnic University, Hong Kong, China,

[email protected]

X. M. YangDepartment of Mathematics, Chongqing Normal University, Chongqing 400047, China, [email protected]

In this paper, a unified framework of a nonlinear augmented Lagrangian dual problem is investigated for the primal problemof minimizing an extended real-valued function by virtue of a nonlinear augmenting penalty function. Our framework is moregeneral than the ones in the literature in the sense that our nonlinear augmenting penalty function is defined on an open setand that our assumptions are presented in terms of a substitution of the dual variable, so our scheme includes barrier penaltyfunctions and the weak peak at zero property as special cases. By assuming that the increment of the nonlinear augmentingpenalty function with respect to the penalty parameter satisfies a generalized peak at zero property, necessary and sufficientconditions for the zero duality gap property are established and the existence of an exact penalty representation is obtained.

Key words : nonlinear augmented Lagrangian; generalized peak at zero; zero duality gap; exact penalty representationMSC2000 subject classification : Primary: 90C30, 90C46, 90C26OR/MS subject classification : Primary: nonlinear programming; secondary: optimizationHistory : Received February 2, 2011; revised August 25, 2011, January 3, 2013. Published online in Articles in Advance

March 29, 2013.

1. Introduction. It is well known that duality theory is always an important issue in optimization. Thezero duality gap property provides a theoretical foundation for establishing the global convergence of numericalalgorithms for solving constrained optimization problems. For convex optimization problems, a zero duality gapproperty is established by using classical Lagrangian functions and assuming some constraint qualifications.For nonconvex optimization problems, the zero duality gap property can be obtained by using augmentedLagrangian functions, in which the augmenting penalty function is the product of the penalty parameter andan augmenting function, and is thus linear in the penalty parameter. Early augmented Lagrangian functionswere usually defined by using convex and quadratic augmenting functions or convex separable augmentingfunctions (see Balder [1], Ben-Tal and Zibulevsky [2], Bertsekas [3], Golshtein and Tretyakov [8], Hestenes [10],Powell [16], Rockafellar [17]). Based on these augmented Lagrangian functions (or their generalized form),many effective numerical algorithms have been designed for solving nonconvex optimization problems (seeBen-Tal and Zibulevsky [2], Bertsekas [3], Burachik et al. [7], Burachik and Kaya [4], Hestenes [10], Luoet al. [12], Polyak [15], Powell [16], Wang and Li [24]).

In the past decade, the study of augmented Lagrangian duality theory has been developed. In Rockafellarand Wets [18], Huang and Yang [11], Rubinov et al. [21], and Rubinov and Yang [20], the primal problem ofminimizing an extended real-valued function, which is more general than a constrained optimization problem,was discussed. In Rockafellar and Wets [18], a nonnegative convex augmenting function and the correspondingaugmented Lagrangian dual problem of the primal problem were introduced. A sufficient condition for the zeroduality gap and a necessary and sufficient condition for the existence of an exact penalty representation wereobtained under mild conditions. In Huang and Yang [11], the same results as Rockafellar and Wets [18] wereobtained by replacing the convexity condition of the augmenting function with a level-boundedness condition.Furthermore, using the theory of abstract convexity (see Rubinov [19]), a family of augmenting functionswith almost peak at zero property and a class of corresponding augmented Lagrangian dual problems wereintroduced in Rubinov et al. [21] and Rubinov and Yang [20]. And it was established that if the family ofaugmenting functions is almost peak at zero, then the lower semicontinuity of the perturbation function at zeroguarantees the zero duality gap property. It is worth noting that the peak at zero property that was employed inRubinov et al. [21] and Rubinov and Yang [20] is more general than the level-boundedness condition in Huangand Yang [11].

Recently, the augmented Lagrangian duality theory has been further advanced in the literature from twoaspects. On one hand, in Nedich and Ozdaglar [14], a nonnegative augmenting function for a primal geometryproblem was introduced and a geometry framework of the corresponding augmented Lagrangian dual problemwas constructed. Under appropriate assumptions necessary and sufficient conditions were obtained for the zero

740

Dow

nloa

ded

from

info

rms.

org

by [

158.

132.

175.

28]

on 1

5 Se

ptem

ber

2015

, at 0

5:12

. Fo

r pe

rson

al u

se o

nly,

all

righ

ts r

eser

ved.

mailto:[email protected]



Wang, Yang, and Yang: Nonlinear Augmented Lagrangian and Duality TheoryMathematics of Operations Research 38(4), pp. 740–760, © 2013 INFORMS 741

duality gap property. It is worth noting that the peak at zero property in Rubinov et al. [21] and Rubinov andYang [20] was further extended in Nedich and Ozdaglar [14], that is, only the nonnegative part of the dualvariable of the augmenting function is required to satisfy a peak at zero property. Obviously, this requirementis weaker than the usual peak at zero property. So this property is called a weak peak at zero property in thepresent paper. In addition, it is noticed that the augmenting penalty function is linear in the penalty parameter inRockafellar and Wets [18], Huang and Yang [11], Rubinov et al. [21], Rubinov and Yang [20] and Nedich andOzdaglar [14]. Thus, if an augmenting function satisfies the weak peak at zero property, then the augmentingpenalty function satisfies the coercivity condition. It is worth noting that if convex augmenting functions inNedich and Ozdaglar [14] are allowed to be negative, the zero duality gap property may not hold. In Nedich andOzdaglar [13], a nonlinear augmenting penalty function, in which the convex augmenting function is allowed tobe negative, was introduced and a geometry framework of the primal and dual problems was constructed. Undervarious coercivity conditions, necessary conditions and sufficient conditions for the zero duality gap propertywere obtained. An exact penalty representation was not discussed in Nedich and Ozdaglar [13, 14], but it waspointed out in Nedich and Ozdaglar [13] that it is significant to study such conditions.

On the other hand, in a Banach space, using the theory of abstract convexity, a framework of an augmentedLagrangian dual problem for the primal problem of minimizing an extended real-valued function was constructedin Burachik and Rubinov [5], in which the nonlinear augmenting penalty function is composite of a couplingfunction with certain monotonic property and augmenting/penalty terms. This framework includes many aug-menting penalty functions (such as the ones in Rockafellar and Wets [18], Huang and Yang [11], Rubinovet al. [21], Rubinov and Yang [20]) as special cases. In Burachik and Rubinov [5], if the family of augmentingfunctions is a peak at zero one and satisfies a coercivity condition, sufficient conditions for the zero duality gapproperty and the existence of an exact penalty representation were given. Later, in a Hausdorff topological space,a unified framework of the augmented Lagrangian dual problem was constructed in Burachik et al. [6], whichis more general than the one in Burachik and Rubinov [5]. It was assumed that the increment of the nonlinearaugmenting penalty function with respect to the penalty parameter satisfies a peak at zero property and a coer-civity condition. In addition, the nonlinear augmenting penalty function was required to be semicontinuous atzero. Under appropriate conditions, a sufficient condition for the zero duality gap and a necessary and sufficientcondition for the existence of an exact penalty representation were obtained in Burachik et al. [6].

Inspired by Burachik et al. [6] and Nedich and Ozdaglar [14], in a finite dimensional Euclidean space, weconstruct a unified framework of the augmented Lagrangian dual problem for the primal problem of minimizingan extended real-valued function by virtue of a nondecreasing nonlinear augmenting penalty function. Thisframework is more general than the ones in Burachik et al. [6] and Nedich and Ozdaglar [14] in a sense thatour nonlinear augmenting penalty function is defined on an open set and that our assumptions are presentedin terms of a substitution of the dual variable, so our scheme includes barrier penalty functions in, e.g., Luoet al. [12], Polyak [15], Sun et al. [22], and the weak peak at zero property used in Nedich and Ozdaglar [14]as special cases. In our assumptions, we need neither the coercivity condition of the increment of the nonlinearaugmenting penalty function as in Burachik et al. [6] or that of the augmenting function as in Nedich andOzdaglar [13], nor the semicontinuity condition of the nonlinear augmenting penalty function at zero as inBurachik et al. [6]. We only assume that the increment of the nonlinear augmenting penalty function with respectto the penalty parameter satisfies a generalized peak at zero property. We introduce a pair of �-perturbationconditions and show that when � > 0 this pair of perturbation conditions present necessary and sufficientconditions for the zero duality gap property; and when �= 0 this pair of perturbation conditions present necessaryand sufficient conditions for the existence of an exact penalty representation. Thus these results clearly reveal theclose relationship between the zero duality gap property and the exact penalty representation property. To the bestof our knowledge, these results have not been investigated in the literature. Furthermore, assuming a coercivitycondition of the increment of the nonlinear augmenting penalty function, we obtain some simplified necessaryconditions and sufficient conditions for the zero duality gap property and the exact penalty representation.

The outline of this paper is as follows. In §2, a nonlinear augmenting penalty function and the correspondingnonlinear augmented Lagrangian dual problem for a primal problem of minimizing an extended real-valuefunction are introduced. In §3, several types of typical nonlinear augmenting penalty functions are presentedand shown to be the special cases of the nonlinear augmenting penalty functions given in this paper. In §4,necessary and sufficient conditions for the zero duality gap property are established. In §5, necessary andsufficient conditions for the existence of an exact penalty representation are obtained.

2. Nonlinear augmented Lagrangian dual problem. Let R̄=R∪ 8+�9. Consider the primal problem ofminimizing an extended real-valued function

4P5 minx∈Rn

�4x51

Dow

nloa

ded

from

info

rms.

org

by [

158.

132.

175.

28]

on 1

5 Se

ptem

ber

2015

, at 0

5:12

. Fo

r pe

rson

al u

se o

nly,

all

righ

ts r

eser

ved.

Wang, Yang, and Yang: Nonlinear Augmented Lagrangian and Duality Theory742 Mathematics of Operations Research 38(4), pp. 740–760, © 2013 INFORMS

where �2 Rn → R̄ is a proper function. A function f̄ 2 Rn ×Rm1 → R̄ is said to be a dualizing parameterizationfunction for �, if f̄ is proper and satisfies

f̄ 4x105= �4x51 ∀x ∈Rn0

The perturbation function of 4P5 is defined as

�f̄ 4u5 2= infx∈Rn

f̄ 4x1 u51 ∀u ∈Rm1 0

The optimal value of 4P5 is denoted by Mp = infx∈Rn �4x5. Then Mp = �f̄ 405.Let R++ = 401+�5. To introduce a nonlinear augmenting penalty function, we introduce a class of auxiliary

set-valued mappings U2 R++ ⇒ Rm1 , which, depending on the penalty parameter r , represents the effectivedomain of the nonlinear augmenting penalty function with respect to the dual variable u. By means of theset-valued mapping U , we are able to include many classical barrier penalty functions as special cases of ournonlinear augmenting penalty function. Assume that U satisfies the following conditions:

(a1) for all r ∈R++, U4r5 is an open and convex set with 0 ∈U4r5; and(a2) for all r1, r2 ∈R++, r2 ≥ r1 ⇒U4r25⊆U4r15.

The following set-valued mappings satisfy conditions (a1) and (a2): for r ∈R++,

U4r5=Rm13

U4r5= 8u ∈Rm1 � �u�< 1/r93

U4r5= 8u ∈Rm1 � ui < 1/r11 ≤ i ≤m193 and

U4r5= 8u ∈Rm1 � ru ∈C91

where C ⊂Rm1 is an open convex set and 0 ∈C.To include the weak peak at zero condition in Nedich and Ozdaglar [14] in condition (A3) of Definition 1

below, we introduce a variable substitution. That is, for a continuous, not identically equal to 0 and nondecreasingfunction e2 R→R with e405= 0, we define the variable substitution �4u5 for u as

�4u5= 4e4u151 : : : 1 e4um1551 u ∈Rm1 0

Some examples of �4u5 follow:(i) If e4t5= t, then �4u5= u= 4u11 : : : 1 um1

5.(ii) If e4t5= t+ 2= max801 t9, then �4u5= u+ = 4u+

1 1 : : : 1 u+m15.

(iii) If e4t5= t− 2= min801 t9, then �4u5= u− = 4u−1 1 : : : 1 u

−m15.

Obviously, u = u+ + u−. We will use the two variable substitutions �4u5 = u and �4u5 = u+, respectively, inour study.

Now we give the definition of a nonlinear augmenting penalty function. Let �2 U4r5×V ×R++ →R, whereU4r5⊆Rm1 satisfies (a1) and (a2) and V ⊆Rm2 is a nonempty subset. For given r , r̄ ∈R++, we denote

ãr̄4u1 v1 r5 2= �4u1 v1 r5−�4u1 v1 r̄50

Definition 1. The function �2 U4r5× V ×R++ → R is called a nonlinear augmenting penalty function ifthe following conditions are satisfied:

(A1) ∀ 4v1 r5 ∈ V ×R++, �401 v1 r5= 0;and there exists a function �4 · 52 Rm1 →Rm1 satisfying (A2) and (A3):(A2) ∀ 4v1 r̄5 ∈ V ×R++,

inf8ãr̄4u1 v1 r5 � u ∈U4r51�4u5= 09≥ 01 ∀ r ≥ r̄ 3

(A3) ∀�> 0, and 4v1 r̄5 ∈ V ×R++,

inf8ãr̄4u1 v1 r5 � u ∈U4r51��4u5� ≥ �9 > 01 ∀ r > r̄0

In (A3), we use the convention inf � = +�.

Dow

nloa

ded

from

info

rms.

org

by [

158.

132.

175.

28]

on 1

5 Se

ptem

ber

2015

, at 0

5:12

. Fo

r pe

rson

al u

se o

nly,

all

righ

ts r

eser

ved.


Remark 1. When �4u5= u, it follows from (A1) that (A2) is always true. Thus � is only required to satisfythe conditions (A1) and (A3) in this case. These two conditions are much weaker than the conditions (C1)and (C2) in Burachik et al. [6]. In fact, the lower semicontinuity of �4·1 v1 r5 at 0 in (C1) and the coercivitycondition of �4u1 v1 ·5 in (C2) are both relaxed in this paper. When �4u1 v1 r5= �u, v� + r�4u5 and �4u5= u+,the condition (A3) becomes the “weak peak at zero condition” of a convex augmenting function in Nedich andOzdaglar [14, Assumption 2(b)], as follows:

inf8�4u5 � �u+� ≥ �9 > 00

That is, for a given sequence 8uk9 ⊂ Rm1 , the convergence of �4uk5 to zero implies the convergence of thenonnegative part of the sequence 8uk9 to zero. Therefore, the condition (A3) is a generalized peak at zerocondition.

Remark 2. Since 8u ∈ U4r5 � �u+� ≥ �9 ⊂ 8u ∈ U4r5 � �u� ≥ �9, any � that satisfies (A1) and (A3) for�4u5= u also satisfies (A1), (A2), and (A3) for �4u5= u+. However the reverse is in general not true.

Using f̄ and � given above, we can construct a nonlinear augmented Lagrangian function of 4P5, that is,

l4x1 v1 r5= infu∈U4r5

8f̄ 4x1u5+�4u1 v1 r591 4x1 v1 r5 ∈Rn×V ×R++0

The dual function of 4P5 is�̄4v1 r5= inf

x∈Rnl4x1 v1 r51 4v1 r5 ∈ V ×R++0

By virtue of the dual function �̄4·1 ·5, we introduce the following dual problem of 4P5:

4D5 sup4v1 r5∈V×R++

�̄4v1 r50

It is easy to see that, for all 4v1 r5 ∈ V ×R++,

�̄4v1 r5= infu∈U4r5

8�f̄ 4u5+�4u1 v1 r590 (1)

Let MD = sup4v1 r5∈V×R++�̄4v1 r5 be the optimal value of the dual problem 4D5. By the definition of f̄ and

conditions (a1) and (A1), it is easy to show that the weak duality property holds, that is,

MD ≤MP or MD ≤ �f̄ 4050

For functions l and �̄, we have the following basic properties.

Proposition 1. For all v ∈ V , we have(i) for all x ∈Rn, l4x1 v1 ·5 is monotonically nondecreasing on r > 0; and

(ii) �̄4v1 ·5 is monotonically nondecreasing on r > 0.

Proof. We only need to prove (i), because (ii) follows from (i) immediately. Let r2 > r1 > 0. From (A2), (A3),for any 4u1 v5 ∈U4r25×V , we have

�4u1 v1 r15≤ �4u1 v1 r250

Noting that U4r25⊆U4r15, for any 4x1 v5 ∈Rn ×V , the above inequality implies that

l4u1 v1 r15 = infu∈U4r15

8f̄ 4x1u5+�4u1 v1 r159

≤ infu∈U4r25

8f̄ 4x1u5+�4u1 v1 r159

≤ infu∈U4r25

8f̄ 4x1u5+�4u1 v1 r259

= l4u1 v1 r250

Thus l4x1 v1 ·5 is monotonically nondecreasing on r > 0. �Proposition 1 is still valid if the strict inequality in (A3) is changed to an inequality. However this strict

inequality is needed in the establishment of the zero duality gap and exact penalty representation properties in§§4 and 5.

Dow

nloa

ded

from

info

rms.

org

by [

158.

132.

175.

28]

on 1

5 Se

ptem

ber

2015

, at 0

5:12

. Fo

r pe

rson

al u

se o

nly,

all

righ

ts r

eser

ved.


Definition 2. We say that the zero duality gap property between 4P5 and its dual problem 4D5 holds if

MD =MP or MD = �f̄ 4050 (2)

Let us consider an important special example of 4P5, that is, the constrained optimization problem

4P̂5 infx∈X0

f 4x51

where X0 = 8x ∈X � gI4x5= 01 gII4x5≤ 09 6= �, X ⊆Rn is a nonempty open subset, gI4x5= 4g14x51 : : : 1 gs4x55T ,

gII4x5= 4gs+14x51 : : : 1 gm14x55T , and f , gj 2 X →R4j = 11 : : : 1m15 are real-valued functions. It is easy to obtain

the nonlinear augmented Lagrangian dual problem of 4P5. Indeed, let

�4x5=

{

f 4x51 if x ∈X01

+�1 otherwise0(3)

Obviously, the constrained optimization problem 4P̂5 is equivalent to 4P5, where � is defined by (3). Let thedualizing parameterization function of 4P̂5 be

f̂ 4x1 u5=

{

f 4x51 if x ∈X1 gI4x5= uI1 gII4x5≤ uII1 u ∈Rm11

+�1 otherwise,(4)

where 4x1u5 ∈ Rn ×Rm1 , u=(

uIuII

)

, uI = 4u11 : : : 1 us5T and uII = 4us+11 : : : 1 um1

5T . So the corresponding pertur-bation function is

�̂f 4u5= inf8f 4x5 � x ∈X1gI4x5= uI1 gII4x5≤ uII91 u ∈Rm1 0 (5)

The corresponding nonlinear augmented Lagrangian penalty function of 4P̂5 is

l̂4x1 v1 r5 = infu∈U4r5

8f̂ 4x1u5+�4u1 v1 r59

= f 4x5+ infu∈U4r5

8�4u1 v1 r5 � x ∈X1gI4x5= uI1 gq4x5≤ uII91 (6)

where � is given in Definition 1. Thus the dual function of 4P̂5 is

�̂4v1 r5= infx∈X

l̂4x1 v1 r51 ∀ 4v1 r5 ∈ V ×R++0 (7)

The dual problem of 4P̂5 is4D̂5 sup

4v1 r5∈V×R++

�̂4v1 r50

In addition, let MP̂ and MD̂ be the optimal values of 4P̂5 and 4D̂5, respectively. Thus all the results establishedfor primal problem 4P5 in this paper can be obtained for the constrained optimization problem 4P̂5 from (3)–(7).These details are omitted because of the page space limitation.

3. Several special types of nonlinear augmenting penalty functions. In this section, we consider severalspecial types of the nonlinear augmenting penalty function introduced in Definition 1, which include manyaugmenting functions and penalty functions in the literature as special cases.

Before presenting examples of the nonlinear augmenting penalty function introduced in Definition 1, we needthe following propositions.

Assume that C ⊆Rm1 is an open convex set with 0 ∈C, and V ⊆Rm2 is a nonempty subset. Let

Rm1+ = 8u ∈Rm1 � u≥ 091

U4r5= 8u ∈Rm1 � ru ∈C91

B = 8u ∈Rm1 � �u� = 191

B+ = 8u ∈Rm1+ � �u� = 190

(8)

Dow

nloa

ded

from

info

rms.

org

by [

158.

132.

175.

28]

on 1

5 Se

ptem

ber

2015

, at 0

5:12

. Fo

r pe

rson

al u

se o

nly,

all

righ

ts r

eser

ved.


Proposition 2. Let P2 C × V → R be a real-valued function such that, for any v ∈ V , P4·1 v5 is a convexfunction on C and P401 v5 = 0. For any � > 0, 4v1 r̄5 ∈ V ×R++ and r > r̄ , let �r ∈ 401 �5 satisfy �rB ⊂ U4r5.Then

infu∈U4r51�u�≥�

{

1rP4ru1 v5−

1r̄P 4r̄u1 v5

}

≥ minu0∈B

{

1rP4r�ru

01 v5−1r̄P 4r̄�ru

01 v5

}

≥ 00

Furthermore, if, for any v ∈ V , P4·1 v5 is a strictly convex function on C, then the second one in the aboveinequalities holds strictly.

Proof. Let v ∈ V . For any r > r̄ > 0 and u ∈U4r5, by the convexity of P4·1 v5 and P401 v5= 0, we have

1rP4ru1 v5−

1r̄P 4r̄u1 v5≥ 00 (9)

Particularly, if P4·1 v5 is strictly convex on C, the inequality (9) holds strictly.For � > 0 and r > r̄ > 0, let u ∈ U4r5 and �r ∈ 401 �5 satisfy �u� ≥ � and �rB ⊂ U4r51 respectively. Let

u= �u�u0 with u0 ∈ B and let ur = �ru0. It follows from the property of U4 · 5 and (8) that ru, r̄u, rur , r̄ur ∈C.

Using the convexity of P4·1 v5 on C, it can be obtained that the difference quotient

1t8P4z+ td1 v5−P4z1 v59

is monotonically nondecreasing in t as long as z1 z + td ∈ C and d ∈ Rm1 . Thus using this property twice,we have

1r̄8P4r̄u1 v5−P4r̄ur 1 v59 =

�u� − �r

r̄ 4�u� − �r58P4r̄ur + r̄ 4�u� − �r5u

01 v5−P4r̄ur 1 v59

≤�u� − �r

r�u� − r̄�r

8P4ru1 v5−P4r̄ur 1 v590

=�u� − �r

r̄�r − r�u�8P4ru+ 4r̄�r − r�u�5u01 v5−P4ru1 v59

≤�u� − �r

r�r − r�u�8P4ru+ 4r�r − r�u�5u01 v5−P4ru1 v59

=1r8P4ru1 v5−P4rur 1 v590

So when u ∈U4r5 and �u� ≥ �, noting ur = �ru0, it follows from rearranging the last inequality that

1rP4ru1 v5−

1r̄P 4r̄u1 v5 ≥

1rP4r�ru

01 v5−1r̄P 4r̄�ru

01 v50 (10)

Now we analyze the right-hand term in (10). Let

�4u01 v5 2=1rP4r�ru

01 v5−1r̄P 4r̄�ru

01 v51 u0∈ B0

By (9), it is clear that �4w1v5 ≥ 0, ∀w ∈ B. It follows from �rB ⊂ U4r5 that for all u0 ∈ B, we haver�ru

01 r̄�ru0 ∈C. From the continuity of convex function P4·1 v5 on the open set C and the compactness of B,

the minimum solution of �4·1 v5 on B exists and

minu0∈B

�4u01 v5≥ 01 v ∈ V 0

Then (10) implies that1rP4ru1 v5−

1r̄P 4r̄u1 v5≥ min

u0∈B�4u01 v50

Since the point u satisfying u ∈U4r5 and �u� ≥ � is arbitrary, we obtain that

infu∈U4r51�u�≥�

{

1rP4ru1 v5−

1r̄P 4r̄u1 v5

}

≥ minu0∈B

�4u01 v50

In particular, if P4·1 v5 is a strictly convex function on C, then by (9) we have �4u01 v5 > 0, ∀u0 ∈ B. Thus

minu0∈B

�4u01 v5 > 01 v ∈ V 0

The proof is complete. �

Dow

nloa

ded

from

info

rms.

org

by [

158.

132.

175.

28]

on 1

5 Se

ptem

ber

2015

, at 0

5:12

. Fo

r pe

rson

al u

se o

nly,

all

righ

ts r

eser

ved.


Proposition 3. Let P2 Rm1 ×V →R be a real-valued function such that, for any v ∈ V , P4·1 v5 is a convexfunction on Rm1 and P401 v5= 0. Let �> 01 �̄ ∈ 401 �5. Then, for any 4v1 r̄5 ∈ V ×R++ and r > r̄ ,

infu∈Rm1 1�u+�≥�

{

1rP4ru+1 v5−

1r̄P 4r̄u+1 v5

}

≥ minu0∈B+

{

1rP4r�̄u01 v5−

1r̄P 4r̄ �̄u01 v5

}

≥ 00

Furthermore, if, for all v ∈ V , P4·1 v5 is a strictly convex function on Rm1+ , then the last one in the above inequalities

holds strictly.

Proof. The results follow from Proposition 2 with C =Rm1 and u+ and B+ instead of u and B, respectively. �Now we use Propositions 2 and 3 to study Examples 1 and 2.

Example 1. Let �4u1 v1 r5= 41/r5P4ru1 v5, 4u1 v1 r5 ∈U4r5×V ×R++ be an augmenting penalty function,where for any v ∈ V , P4·1 v5 is strictly convex on the open convex set C ⊆Rm1 with 0 ∈C, P401 v5= 0, U4r5 isgiven by (8), and V ⊂Rm2 is a nonempty subset. Let �4u5= u.

We check that � satisfies the conditions (A1) and (A3) in Definition 1. First, from P401 v5 = 0, it is easyto check that � satisfies the condition (A1). To verify condition (A3), by the strict convexity of P4·1 v5 andProposition 2, we have that, for �> 0,

inf84r̄4u1 v1 r5 � u ∈U4r51�u� ≥ �9 > 01 ∀ 4v1 r̄5 ∈ V ×R++1 ∀ r > r̄0

Hence, � satisfies (A3). Therefore, this � is a special case of the nonlinear augmenting penalty function introducedin Definition 1.

We will use (R1)–(R5) to denote various comparisons between the special cases of our nonlinear augmentingpenalty function presented in Examples 1–3 and those in the literature.

(R1) It is known that the nonlinear augmenting penalty function of the form in Example 1 and its special casesare widely used in the literature, in which, however apart from the convexity or strict convexity of P4·1 v5, somestrong conditions (such as coercivity condition, continuous differentiability condition, and so on) are assumed.Notice that P4·1 v5 is only required to be strictly convex on C in Example 1. So, P4·1 v5 in Example 1 is moregeneral than other augmenting penalty functions in the literature.

In Nedich and Ozdaglar [13], an augmented Lagrangian dual problem of a geometric primal problem wasintroduced by the following nonlinear augmenting penalty function:

1r�4ru5 ∀ 4u1 r5 ∈Rm1 ×R++1 (11)

where the augmenting function �2 Rm1 → R̄ is proper and convex on Rm1 , not identically equal to 0, and �405=

0. When � satisfies three types of the coercivity condition (see Nedich and Ozdaglar [13, Assumptions 3.2, 3.3,and 3.6]), respectively, necessary and sufficient conditions for the zero duality gap were obtained.

In the following we show that (11) is actually a special case of Example 1. Let the effective domain C of� , denoted by dom(�), be open. It follows from the convexity of � and �405 = 0 that C is an open convexset in Rm1 and 0 ∈C. For example, the effective domain of the augmenting function � satisfying in Nedich andOzdaglar [13, Assumption 3.6(a)] is such an open convex set. Let U4 · 5 be given by (8). Thus (11) is a specialcase of Example 1, that is, V = 809 and

�4u101 r5=1r�4ru5 ∀ 4u1 r5 ∈U4r5×R++0 (12)

The following augmenting functions satisfy in Nedich and Ozdaglar [13, Assumption 3.3]:

�i4u5=

m1∑

i=1

�i4ui51 i = 112131

where

�14t5=

{

− log41 − t51 if t < 13+�1 otherwise1

dom4�15=C = 8u ∈Rm1 � ui < 111 ≤ i ≤m191

�24t5=

{

t/41 − t51 if t < 13+�1 otherwise1

dom4�25=C = 8u ∈Rm1 � ui < 111 ≤ i ≤m191

�34t5=

{

t + 12 t

21 if t ≥ −12 3

−12 log4−2t5−

38 1 otherwise1

dom4�35=C =Rm11

respectively.

Dow

nloa

ded

from

info

rms.

org

by [

158.

132.

175.

28]

on 1

5 Se

ptem

ber

2015

, at 0

5:12

. Fo

r pe

rson

al u

se o

nly,

all

righ

ts r

eser

ved.


The above three augmenting functions �i 4i = 112135 are strictly convex on open sets C. Therefore, theaugmenting penalty functions (12) constructed by using �i 4i = 112135 are the special cases of Example 1.Moreover, there are still many strictly convex augmenting functions that do not belong to any of the three typesof augmenting functions in Nedich and Ozdaglar [13]. Two strictly convex functions below (given in Gonzagaand Castillo [9]) are such examples:

�44t5= log1 + et

21 t ∈R1

�54t5=t +

√t2 + 42

− 11 t ∈R0

As such, under the strict convexity condition of � , the nonlinear augmenting penalty function (11) introducedin Nedich and Ozdaglar [13] is a special case of Example 1. It is worth noting that the five nonlinear augmentingpenalty functions 41/r5�i4ru5 4i = 1121 : : : 155 mentioned above do not satisfy any of the coercivity conditionsin Burachik and Rubinov [5] and Burachik et al. [6].

(R2) Let us consider the application of Example 1 to the optimization problem 4P̂5. As will be shown, wewill derive several important types of augmenting penalty functions by some special forms of Example 1. Firsttake the following special case of Example 1:

�4u1 v1 r5=v

r

m1∑

i=1

�4rui51 4u1 v1 r5 ∈Rm1 ×R++ ×R++1 (13)

where m2 = 1, � is monotonically nondecreasing and strictly convex on R with �405 = 0. Substituting (13)into (6), we obtain the following augmented Lagrangian function of 4P̂5:

l̂4x1 v1 r5= f 4x5+v

r

m1∑

i=1

�4rgi4x551 4x1 v1 r5 ∈X ×R++ ×R++0 (14)

If furthermore � is continuously differentiable, limt→−� �4t5 = a ∈ R and limt→+�4�4t5/t5 = 1 (it is easy toverify that �i4t54i = 4155 satisfy these conditions), then the augmented Lagrangian function in (14) is the onegiven in Gonzaga and Castillo [9], which can smoothly approximate the l1 exact penalty function, that is,

limr→+�

v

r

m1∑

i=1

�4rgi4x55= vm1∑

i=1

max801 gi4x590

Now we take another special case of Example 1:

�4u1 v1 r5=1r

m1∑

i=1

vi�4rui51 4u1 v1 r5 ∈Rm1 ×Rm1++ ×R++1 (15)

where � is monotonically nondecreasing and strictly convex on R with �405= 0. Substituting (15) into (6), weobtain another augmented Lagrangian function of 4P̂5:

l̂4x1 v1 r5= f 4x5+1r

m1∑

i=1

vi�4rgi4x551 4x1 v1 r5 ∈X ×Rm1++ ×R++0 (16)

In addition, if � is twice continuously differentiable with �′405 = 1, �′′405 > 0, limt→−� �4t5 > −�,limt→−� �′4t5= 0, and limt→+� �′4t5= +�, then the augmented Lagrangian function (16) is a P̂I penalty func-tion in Bertsekas [3], a widely used exponential type augmenting penalty function in the literature (see Luoet al. [12], Sun et al. [22], Tseng and Bertsekas [23], and Wang et al. [25]).

Finally, we take the following special case of Example 1:

�4x1 v1 r5= �u1 v� +1r

m1∑

i=1

�4rui51 4u1 v1 r5 ∈Rm1 ×Rm1 ×R++1 (17)

where � is strictly convex on R with �405= 0. Substituting (17) into (6), it follows that

l̂4x1 v1 r5 = f 4x5+1r

m1∑

i=1

infgi4x5≤ui

8ruivi +�4rui59

= f 4x5+1r

m1∑

i=1

�4rgi4x51 vi51 4x1 v1 r5 ∈X ×Rm1 ×R++1 (18)

Dow

nloa

ded

from

info

rms.

org

by [

158.

132.

175.

28]

on 1

5 Se

ptem

ber

2015

, at 0

5:12

. Fo

r pe

rson

al u

se o

nly,

all

righ

ts r

eser

ved.


where�4s1 �5= inf

s≤t8t� +�4t590 (19)

In addition, if � is continuously differentiable with �′405= 0, limt→−� �′4t5= −�, and limt→+� �′4t5= +�,then (19) can be expressed by

�4s1 �5=

{

s� +�4s51 if � +�′4s5≥ 03

mint∈R

8t� +�4t591 otherwise0

In this case, (18) is a P+

E type penalty function in Bertsekas [3]. If � is also twice continuously differentiablewith �′′405 > 0, then l̂4x1 v1 r5 in (18) is the essential twice augmenting penalty function widely used in theliterature (see Luo et al. [12], Rockafellar [17], Rockafellar and Wets [18], Sun et al. [22], Wang et al. [25], andWang and Li [24]).

Similarly, we can also use other special cases of Example 1 to derive exponential type (Bertsekas [3],Polyak [15], Sun et al. [22]) and barrier type (Luo et al. [12], Polyak [15], Sun et al. [22]) penalty functionsfor 4P̂5. These details are omitted here.

Example 2. Let �4u1 v1 r5= 41/r5P4ru1 v5, 4u1 v1 r5 ∈Rm1 ×V ×R++1 where P2 Rm1 ×V →R satisfies thefollowing conditions:

(p1) for any v ∈ V , P4·1 v5 is convex on Rm1 and strictly convex on Rm1+ and P401 v5= 03

(p2) for all 4u1 v1 r̄5 ∈Rm1 ×V ×R++ and r > r̄ , there holds

1rP4ru1 v5−

1r̄P 4r̄u1 v5≥

1rP4ru+1 v5−

1r̄P 4r̄u+1 v50

Let �4u5 = u+. We verify that the function � satisfies conditions (A1), (A2), and (A3) of Definition 1. First,from P401 v5= 0 and (9), it is easy to check that � satisfies (A1) and (A2). Next we verify that � satisfies (A3).Let � > 00 From the strict convexity of P4·1 v5 on R

m1+ 1 the assumption (p2) and Proposition 3, it follows that,

for all 4v1 r̄5 ∈ V ×R++ and r > r̄ ,

inf8ãr̄4u1 v1 r5 � u ∈Rm11�u+� ≥ �9≥ inf

u∈Rm1 1�u+�≥�

{

1rP4ru+1 v5−

1r̄P 4r̄u+1 v5

}

> 00

So (A3) holds. Therefore, Example 2 is a special case of the nonlinear augmenting penalty function introducedin Definition 1.

Now we consider two special cases of Example 2.(R3) Special case 1 of Example 2. Let

�4u1 v1 r5=1r

m1∑

i=1

ê4rui1 vi51 4u1 v1 r5 ∈Rm1 ×Rm1+ ×R++1 (20)

where P4u1 v5 =∑m1

i=1 ê4ui1 vi5, ê2 R × R+ → R satisfies the following condition: for all s ∈ R+, ê4·1 s5 isconvex on Rm1 and strictly convex on R

m1+ with ê401 s5= 0.

Obviously, P satisfies (p1). Next we verify that P satisfies (p2). Let 4u1 v1 r̄5 ∈ Rm1 ×Rm1+ ×R++ and r > r̄ .

By (9) and ê401 s5= 01 we have,

1rP4ru1 v5−

1r̄P 4r̄u1 v5

=∑

ui≥0

{

1rê4rui1 vi5−

1r̄ê4r̄ui1 vi5

}

+∑

ui<0

{

1rê4rui1 vi5−

1r̄ê4r̄ui1 vi5

}

≥∑

ui≥0

{

1rê4ru+

i 1 vi5−1r̄ê4r̄u+

i 1 vi5

}

+∑

ui<0

{

1rê4ru+

i 1 vi5−1r̄ê4r̄u+

i 1 vi5

}

=1rP4ru+1 v5−

1r̄P 4r̄u+1 v50

So (p2) holds.

Dow

nloa

ded

from

info

rms.

org

by [

158.

132.

175.

28]

on 1

5 Se

ptem

ber

2015

, at 0

5:12

. Fo

r pe

rson

al u

se o

nly,

all

righ

ts r

eser

ved.


Now we consider the application of special case 1 of Example 2 (that is, � is defined by (20)) to optimizationproblem 4P̂5. Let ê4·1 s5 be monotonically nondecreasing for any s ∈ R+. Substituting � of (20) into (6), weobtain the following augmented Lagrangian function of 4P̂5:

l̂4x1 v1 r5= f 4x5+1r

m1∑

i=1

ê4rgi4x51 vi51 4x1 v1 r5 ∈X ×Rm1+ ×R++1

where m2 = m1. In addition, if ê is continuously differentiable on R × R++ and satisfies some coercivityconditions (see Bertsekas [3, pp. 305–308]), then P is a P̂I type penalty function in Bertsekas [3].

(R4) Special case 2 of Example 2. Let V = 809 and

�4u101 r5=1r�4ru51 4u1 r5 ∈Rm1 ×R++1 (21)

where P = �2 Rm1 →R satisfies conditions (p1) and (p2). By (20), we know that one subclass of (21) is

�4u5=

m1∑

i=1

�4ui51 (22)

where �2 R→R is convex on R and strictly convex on R+ with �405= 00 Obviously (22) is more general thanthe case that � is a strictly convex function on R.

It is worth noting that the real-valued convex augmenting function � in (21) contains some examples of thefirst type (satisfying Nedich and Ozdaglar [13, Assumption 3.2]) and the second type (satisfying Nedich andOzdaglar [13, Assumption 3.3]) augmenting functions. For example, (see, e.g., Nedich and Ozdaglar [13, 14],Bertsekas [3]):�14u5= 4u+5TQu+1 where Q is a symmetric positive definite matrix;�24u5= max801 a14e

u1 − 151 : : : 1 am4eum1 − 1591 where ai > 0, 1 ≤ i ≤m1;

�34u5=∑m1

i=14max801 ui95�, where �> 1;

�44u5=∑m1

i=1 �4ui5, where

�4t5=

{

t + 12 t

21 if t ≥ 01

− log41 − t51 otherwise0

Of course, there are also augmenting functions � of (21), which do not belong to the cases of Nedich andOzdaglar [13]. Such examples are

�54u5= max8− log41 + u+

1 51 : : : 1− log41 + u+

m1590

�64u5=

m1∑

i=1

�4ui51 �4t5=

{

e−t − 11 if t ≥ 01

−t1 otherwise0

It is worth noting that if the augmenting function � in (21) satisfies either Nedich and Ozdaglar [13, Assump-tion 3.2(b)] or Nedich and Ozdaglar [13, Assumption 3.3(a)], then, by (p2) and Proposition 3, we can easilyprove that, for all �> 01

infu∈Rm1 1�u+�≥�

ãr̄4u101 r5 2= infu∈Rm1 1�u+�≥�

{

1r�4ru5−

1r̄�4r̄u5

}

≥ infu∈Rm1 1�u+�≥�

{

1r�4ru+5−

1r̄�4r̄u+5

}

→ +� 4as r → +�50

That is to say, these two assumptions in Nedich and Ozdaglar [13] are too strong.

Example 3. Let �2 Rm1 × V ×R++ →R, 4u1 v1 r5 ∈Rm1 × V ×R++1 where V = V1 × V2 ⊆Rm2 satisfies thefollowing conditions:

(�1) for all 4v1 r5 ∈ V ×R++, �401 v1 r5= 0;(�2) for all 4u1 v11 v21 r̄5 ∈Rm1 ×V1 ×V2 ×R++ and r ≥ r̄ , there holds

ãr̄4u1 v1 r5≥ ˜�4�v24u51 r1 r̄51

where �̃2 R+ ×R++ ×R++ →R is any function with the following properties:(�̃1) for all 4r1 r̄5 ∈R++ ×R++, �̃401 r1 r̄5= 0;(�̃2) when r > r̄ , �̃4·1 r1 r̄5 is strictly increasing,

Dow

nloa

ded

from

info

rms.

org

by [

158.

132.

175.

28]

on 1

5 Se

ptem

ber

2015

, at 0

5:12

. Fo

r pe

rson

al u

se o

nly,

all

righ

ts r

eser

ved.


and the family of functions 8�v29v2∈V2

has the following properties:(�1) for all 4u1 v25 ∈Rm1 ×V2, �v2

4u5≥ 0 and �v2405= 0;

(�2) there exists a variable substitution �4u5 for u such that, for all �> 0 and v2 ∈ V2,

�v21 �2= inf

u∈Rm1 1��4u5�≥��v2

4u5 > 00

Based on the above conditions, we can easily verify that � satisfies conditions (A1), (A2), and (A3) in Definition 1.

(R5) Let � be given by the conditions (3.1)–(3.3), 4U251 4U45, and 4Y15 in Burachik and Rubinov [5], �4u5= u1and, when r ≥ r̄ > 0, �̃4t1 r1 r̄5 = �44r − r̄ 5t5, where � satisfies condition (3.1) in Burachik and Rubinov [5].Then −� is a special case of the nonlinear augmenting penalty function of Example 3.

In the following, we give a concrete example of Example 3, but it does not belong to the augmenting penaltyfunction in Burachik and Rubinov [5]. Let

�4u1 v1 r5= �u1 v� + log41 + rq�v24u551 4u1 v1 r5 ∈Rm1 ×V ×R++1 (23)

where V = V1 ×V2 ⊆Rm1 , q ∈ 401+�5, 8�v29v2∈V2

satisfies 4�15 and, for �4u5= u+, 8�v29v2∈V2

satisfies 4�25.It is clear that � defined in (23) satisfies 4�15. We only need to verify that the function � satisfies 4�25. Let

4u1 v1 r̄5 ∈ Rm1 × V ×R++ and r > r̄ . By the mean-value theorem and the monotonicity of s/41 + s5 on s > 01we have,

ãr̄4u1 v1 r5 = log41 + rq�v24u55− log41 + r̄q�v2

4u55

=qr̂q−1�v2

4u5

1 + r̂q�v24u5

4r − r̄ 5 4r̄ < r̂ < r5

≥qr̄q�v2

4u5

r41 + r̄q�v24u55

4r − r̄ 5

= �̃4�v24u51 r1 r̄51

where �̃4t1 r1 r̄5= q41 − r̄/r5r̄qt/41 + r̄qt5 obviously satisfies 4�̃15 and 4�̃25.

4. Zero duality gap property. In this section, we present some necessary and sufficient conditions for thezero duality gap property (2) to hold, and obtain some corollaries, some of which are known results, and someothers provide a theoretical foundation for the global convergence of the augmented Lagrangian method. Let

N4v1 r1 �5= 8u ∈U4r5 � �f̄ 4u5+�4u1 v1 r5≤ �91

where U4 · 5 satisfies (a1) and (a2). Let

W �� 405= 8u ∈Rm1 � ��4u5� ≤ �91 � > 00

It is clear from the nondecreasing property of e that W �� 405 is a neighborhood of 0 ∈ Rm1 . In particular,

if �4u5= u, it is denoted by W�405. If �4u5= u+, it is denoted by W+� 405.

We now give necessary and sufficient conditions for the zero duality gap property to hold.Recall that �̄4v1 r5 is the dual function of 4P5. Assume that, for any �> 0 and r̄ > 0, there exist r� > r̄ , v� ∈ V

and a neighborhood W �� 405 of 0 ∈Rm1 such that

4B�15 inf8ãr̄4u1 v�1 r�5 � u ∈N4v�1 r�1 �+ �59= 0, ∀�> �̄4v�1 r�5;

4B�25 �f̄ 405− �≤ �f̄ 4u5+�4u1 v�1 r�5, ∀u ∈W �

� 405∩U4r�5;4B�

15 and 4B�25 are called a set of the �-perturbation conditions.

Theorem 1. Assume that the nonlinear augmenting penalty function � satisfies 4A15, 4A25, and 4A35. Thenthe zero duality gap property (2) holds if and only if 4B�

15 and 4B�25 hold.

Proof. Necessity. Let MP = MD. Notice that �̄4v1 ·5 is monotonically nondecreasing on r > 0. Then, forany �> 0 and r̄ > 0, there exist r� > r̄ and v� ∈ V such that

�f̄ 405− � ≤ �̄4v�1 r�5

= infu∈U4r�5

8�f̄ 4u5+�4u1 v�1 r�59

≤ �f̄ 4u5+�4u1 v�1 r�51 ∀u ∈U4r�50 (24)

Dow

nloa

ded

from

info

rms.

org

by [

158.

132.

175.

28]

on 1

5 Se

ptem

ber

2015

, at 0

5:12

. Fo

r pe

rson

al u

se o

nly,

all

righ

ts r

eser

ved.


From (24), we know that 4B�25 holds. Let �> �̄4v�1 r�5. When u ∈N4v�1 r�1 �5, (A1) and (24) imply that

�f̄ 405+�401 v�1 r�5 = �f̄ 405

≤ �f̄ 4u5+�4u1 v�1 r�5+ �

≤ �+ �0

That is, 0 ∈N4v�1 r�1 �+ �5. By using 4A15, 4A25, and 4A35, we have

inf8ãr̄4u1 v�1 r�5 � u ∈N4v�1 r�1 �+ �59= 01 ∀�> �̄4v�1 r�50

So 4B�15 holds.

Sufficiency. We assume that, for any � > 0 and r̄ > 0, there exist r� > r̄ and v� ∈ V and a neighborhoodW �

� 405 of 0 ∈Rm1 satisfying 4B�15 and 4B�

25. Choose a sequence 8�k9 such that

�k ↓ �̄4v�1 r�5 4as k → +�50 (25)

By condition 4B�15, there exists a sequence 8uk9⊂N4v�1 r�1 �k + �5 such that

limk→+�

ãr̄4uk1 v�1 r�5= 00 (26)

By (26) and (A3), we have thatlim

k→+��4uk5= 00

Hence, when k is sufficiently large, we have uk ∈ W �� 405∩U4r�5. Noting that uk ∈ N4v�1 r�1 �k + �5, together

with 4B�25, we have

�f̄ 405− � ≤ �f̄ 4uk5+�4uk1 v�1 r�5

≤ �k + �0

By the inequality above and (25), we have

�f̄ 405− � ≤ limk→+�

�k + �

= �̄4v�1 r�5+ �

≤ MD + �0 (27)

Since � > 0 is arbitrary, (27) and the weak duality property yield that the zero duality gap property holds, thatis, MP =MD. �

Remark 3. Theorem 1 illustrates that 4B�15 and 4B�

25 are necessary and sufficient conditions for the zeroduality gap property to hold. This set of perturbation conditions will be shown to be a necessary and sufficientcondition for the exact penalty representation of 4P5 when � = 0 (see Theorem 3). Thus this set of the pertur-bation conditions clearly reveals the close relation between the zero duality gap property and the exact penaltyrepresentation. To the best of our knowledge, no such kind of perturbation conditions have been investigated inthe literature yet.

Let us introduce a coercivity assumption as follows:(A4) for all �> 0 and 4v1 r̄5 ∈ V ×R++, we have

lim infr→+�

8ãr̄4u1 v1 r5 � u ∈U4r51��4u5� ≥ �9= +�0

When U4r5=Rm1 and �4u5= u, (A4) reduces to Assumption (C2)-(ii) in Burachik et al. [6].We have the following results.

Lemma 1. Let �̄ >MP . Suppose that the assumption (A4) holds, and there exists 4v̄1 r̄5 ∈ V ×R++ satisfying

�̄4v̄1 r̄5 >−�0

Then, for any � > 0, there exists r� ≥ r̄ such that

N4v̄1 r�1 �5⊆W �� 405∩U4r�51 ∀� ∈ 4�̄4v̄1 r�51 �̄70

Dow

nloa

ded

from

info

rms.

org

by [

158.

132.

175.

28]

on 1

5 Se

ptem

ber

2015

, at 0

5:12

. Fo

r pe

rson

al u

se o

nly,

all

righ

ts r

eser

ved.


Proof. Assume to the contrary that there exists �̄ > 0, for all r > r̄ , we can choose �r ∈ 4�̄4v̄1 r51 �̄7 such that

ur ∈N4v̄1 r1 �r51 (28)

but ur 6∈W ��̄ 405. It follows from ur 6∈W �

�̄ 405 that

��4ur5�> �̄0 (29)

From (28), ur ∈U4r5⊆U4r̄5, for all r > r̄ , and thus we have, for all r > r̄ ,

�̄ ≥ �r

≥ �f̄ 4ur5+�4ur 1 v̄1 r5

= �f̄ 4ur5+�4ur 1 v̄1 r̄5+ãr̄4ur 1 v̄1 r50

Furthermore, from (1) and ur ∈U4r̄5, we have

�̄≥ �̄4v̄1 r̄5+ãr̄4ur 1 v̄1 r50

This, together with (29) and ur ∈U4r5, we have

�̄≥ �̄4v̄1 r̄5+ inf8ãr̄4u1 v̄1 r5 � u ∈U4r51��4u5� ≥ �̄90 (30)

As �̄4v̄1 r̄5 >−�1 by (A4), the formula (30) deduces a contradiction. �Corollary 1. Suppose that the assumptions of Theorem 1 hold, (A4) holds, and there exists 4v̄1 r̄5 ∈

V ×R++ such that�̄4v̄1 r̄5 >−�0

If, for all �> 01 there exists a neighborhood W �� 405 of 0 ∈Rm1 such that

�f̄ 405− �≤ �f̄ 4u5+�4u1 v̄1 r̄51 ∀u ∈W �� 405∩U4r̄51 (31)

then the zero duality gap (2) holds.

Proof. We only need to prove that 4B�15 and 4B�

25 hold. For all � > 01 there exists a neighborhood W �� 405

of 0 ∈Rm such that 4315 holds. Notice that � depends on � in 4315. Thus, for this � > 0, Lemma 1 implies thatthere exists r� ≥ r̄ such that

N4v̄1 r�1 �5⊆W �� 405∩U4r�51 ∀� ∈ 4�̄4v̄1 r�51 �̄70 (32)

From U4r�5⊆U4r̄5, the monotonic nondecreasing property of � on r > 0 and 4315, we have

�f̄ 405− � ≤ �f̄ 4u5+�4u1 v̄1 r̄5

≤ �f̄ 4u5+�4u1 v̄1 r�51 ∀u ∈W �� 405∩U4r�50 (33)

This shows that 4B�25 holds. From (32) and (33), we have, for all u ∈N4v̄1 r�1 �5,

�f̄ 405− �≤ �f̄ 4u5+�4u1 v̄1 r�5≤ �1 ∀� ∈ 4�̄4v̄1 r�51 �̄70 (34)

It follows from (A1) and (34) that 0 ∈N4v̄1 r�1 �+ �5. So by (A1), (A2), and (A3), we have

inf8ãr̄4u1 v̄1 r�5 � u ∈N4v̄1 r�1 �+ �59= 01 ∀� ∈ 4�̄4v̄1 r�51 �̄70

Thus 4B�15 holds. �

Definition 3. Let �4u5 be a variable substitution for u. Then f 2 Rm1 → R is said to be �-lower semicon-tinuous at u= 0 if

lim inf�4u5→0

f 4u5≥ f 4050

Similarly, the �-upper semicontinuity of f at u= 0 can be defined. We say that the function f is �-continuousat u= 0, if

lim�4u5→0

f 4u5= f 4050

Dow

nloa

ded

from

info

rms.

org

by [

158.

132.

175.

28]

on 1

5 Se

ptem

ber

2015

, at 0

5:12

. Fo

r pe

rson

al u

se o

nly,

all

righ

ts r

eser

ved.


Obviously, when �4u5 = u, the �-continuity (�-semicontinuity) of f at u = 0 is equivalent to continuity(semicontinuity) of f at u= 0. When �4u5= u+, the �-continuity (�-semicontinuity) of f at u= 0 implies thatf is continuous (semicontinuous) at u = 0. But the reverse is in general not true. Simple counterexamples canbe constructed easily. Details are not given here.

Corollary 2. Suppose that the assumptions of Theorem 1 hold, 4A45 holds, and there exists 4v̄1 r̄5 ∈

V ×R++ such that�̄4v̄1 r̄5 >−�0

Then(i) if �4·1 v̄1 r̄5 and �f̄ 4 · 5 are �-lower semicontinuous at u= 0, then the zero duality gap (2) holds;

(ii) let the zero duality gap (2) hold. If, for all 4u1 r5 ∈ V × R++, �4·1 v1 r5 is �-continuous at u = 0, then�f̄ 4 · 5 is �-lower semicontinuous at u= 0.

Proof. (i) Assume that �4·1 v̄1 r̄5 and �f̄ 4 · 5 are �-lower semicontinuous at u= 0. Then for all �> 01 thereexists a neighborhood W �

� 405 of 0 ∈Rm such that 4315 holds. By Corollary 1, we know that (i) holds.(ii) Now, let MP = MD. By Theorem 1, for all � > 01 4B�

25 holds. By the �-continuity of �4·1 v1 r5 at u = 0and 4B�

25, we get that �f̄ 4 · 5 is �-lower semicontinuous at u= 0. �

From Corollary 2(i) and the fact that (A2) always holds when �4u5= u, we have the following result.

Corollary 3. Assume that there exists 4v̄1 r̄5 ∈ V ×R++ such that

�̄4v̄1 r̄5 >−�1

that � satisfies 4A15, 4A35, and 4A4) with �4u5 = u and that �4·1 v̄1 r̄5 and �f̄ 4 · 5 are lower semicontinuous atu= 0. Then the zero duality gap property (2) holds.

Remark 4. Corollary 3 becomes Theorem 3.2 of Burachik et al. [6], when U4r5=Rm1 and −� replaces �,apart from the fact that �4·1 v̄1 r̄5 is only required to be lower semicontinuous at 0 in our corollary. However,−�4·1 y1 r5 was required to be lower semicontinuous at 0 for any 4y1 r5 ∈ Y × R+ in (C1) of Theorem 3.2 ofBurachik et al. [6].

By Proposition 1, the dual function �̄4v1 ·5 of 4P5 is monotonically nondecreasing on r > 0. Hence,limr→+� �̄4v1 r5 exists (finite or +�). In the following, we present a necessary and sufficient condition for thislimit to coincide with the optimal value �f̄ 405 of 4P5. This result provides a theoretical foundation for the globalconvergence of an augmented Lagrangian method of 4P5 by using the function l.

Lemma 2. Suppose that the assumptions of Theorem 1 hold, 4A45 holds, and there exists 4v̄1 r̄5 ∈ V ×R++

such that�̄4v̄1 r̄5 >−�0

If �4·1 v̄1 r̄5 is �-lower semicontinuous at u= 0, then

lim inf�4u5→0

�f̄ 4u5≤ limr→+�

�̄4v̄1 r50 (35)

Furthermore, if for all r > 01 �4·1 v̄1 r5 is �-continuous at u= 0, then (35) holds as an equality.

Proof. By the assumptions and the monotonic nondecreasing property of �̄4v1 ·5 on r > 0, we have

�̄4v̄1 r5 >−�1 ∀ r ≥ r̄ 0

Noting that�̄4v̄1 r5= inf

u∈U4r58�f̄ 4u5+�4u1 v̄1 r591

then, from Lemma 1, for �k ↓ 0 (as k → +�51 there exist rk ↑ +� (as k → +�5 and uk ∈U4rk5 such that

�f̄ 4uk5+�4uk1 v̄1 rk5≤ �̄4v̄1 rk5+�k 2= �k1 (36)

anduk ∈N4v̄1 rk1 �k5⊆W �

�k405∩U4rk50 (37)

Dow

nloa

ded

from

info

rms.

org

by [

158.

132.

175.

28]

on 1

5 Se

ptem

ber

2015

, at 0

5:12

. Fo

r pe

rson

al u

se o

nly,

all

righ

ts r

eser

ved.


By (37), we havelim

k→+��4uk5= 00 (38)

For rk ≥ r̄ , we know that U4rk5⊆U4r̄5. Hence, from (A1), (A2), and (A3), we have

�4uk1 v̄1 rk5≥ �4uk1 v̄1 r̄50 (39)

By (36) and (39), we have

�̄4v̄1 rk5+�k ≥ �f̄ 4uk5+�4uk1 v̄1 rk5

≥ �f̄ 4uk5+�4uk1 v̄1 r̄50

Since �4·1 v̄1 r̄5 is �-lower semicontinuous at u= 0, by (A1), (38), and the inequality above, we have

lim inf�4u5→0

�f̄ 4u5 ≤ lim inf�4uk5→0

�f̄ 4uk5

≤ lim inf�4uk5→0

�f̄ 4uk5+ lim inf�4uk5→0

�4uk1 v̄1 r̄5

≤ lim inf�4uk5→0

8�f̄ 4uk5+�4uk1 v̄1 r̄59

≤ limr→+�

�̄4v̄1 r50

So (35) holds. Assume now that, for all r > 0, �4·1 v̄1 r5 is �-continuous at u= 0. As such, by

�̄4v̄1 r5≤ �f̄ 4u5+�4u1 v̄1 r51 ∀u ∈U4r51

combining the �-continuity of �4·1 v̄1 r5 at u= 0 with (A1), we have that, for all r > 0,

�̄4v̄1 r5 ≤ lim inf�4u5→0

8�f̄ 4u5+�4u1 v̄1 r59

= lim inf�4u5→0

�f̄ 4u50

Hence,lim

r→+��̄4v̄1 r5≤ lim inf

�4u5→0�f̄ 4u50

By (35) again, the equality in (35) holds. �In the following, we present necessary and sufficient conditions for the following equality to hold:

limr→+�

�̄4v̄1 r5= �f̄ 4050 (40)

Theorem 2. Suppose that the assumptions of Theorem 1 hold, 4A45 holds, and there exists 4v̄1 r̄5 ∈ V ×R++

such that�̄4v̄1 r̄5 >−�0

Then(i) if �4·1 v̄1 r̄5 and �f̄ 4 · 5 are �-lower semicontinuous at u= 0, then (40) holds;

(ii) assume that (40) holds. If, for all 4v1 r5 ∈ V × R++, �4·1 v1 r5 is �-continuous at u = 0, then �f̄ 4 · 5 is�-lower semicontinuous at u= 0.

Proof. (i) Assume that �4·1 v̄1 r̄5 and �f̄ 4 · 5 are �-lower semicontinuous at u = 0. Then Lemma 2 impliesthat (37) holds. Hence, we have

�f̄ 405≤ lim inf�4u5→0

�f̄ 4u5≤ limr→+�

�̄4v̄1 r5≤MD0 (41)

Then by the weak duality property and (41), (40) holds.(ii) Now we assume that (40) holds and, for all 4v1 r5 ∈ V ×R++, �4·1 v1 r5 is �-continuous at u= 0. By the

weak duality property and (40), the zero duality gap property (2) holds. Thus Corollary 2(ii) implies that �f̄ 4 · 5is �-lower semicontinuous at u= 0. �

Dow

nloa

ded

from

info

rms.

org

by [

158.

132.

175.

28]

on 1

5 Se

ptem

ber

2015

, at 0

5:12

. Fo

r pe

rson

al u

se o

nly,

all

righ

ts r

eser

ved.


5. Exact penalty representation. In this section, we give a necessary and sufficient condition for an exactpenalty representation and show that some corollaries of this condition include several recent results in theliterature as special cases.

Definition 4. A vector v̄ ∈ V is said to support an exact penalty representation of 4P5 if there exists r̄ ∈R++

such that

�f̄ 405= infx∈Rn

l4x1 v̄1 r51 ∀ r ≥ r̄ 1 (42)

arg minx∈Rn

�4x5= arg minx∈Rn

l4x1 v̄1 r51 ∀ r ≥ r̄ 0 (43)

A necessary and sufficient condition for v̄ to support an exact penalty representation of 4P5 can be obtainedfrom the �-perturbation conditions 4B�

15 and 4B�25 when setting �= 0, v̄ = v0 and r̄ = r0. Hence, these conditions

are presented as follows.There exist r̄ ∈R++ and a neighborhood W �

� 405 of 0 ∈Rm1 such that4B0

15 inf8ãr̄4u1 v̄1 r5 � u ∈N4v̄1 r1 �59= 0, ∀ r ≥ r̄ 1 � > �̄4v̄1 r5 and4B0

25 �f̄ 405≤ �f̄ 4u5+�4u1 v̄1 r̄5, ∀u ∈W �� 405∩U4r̄5.

In the following, we establish that 4B015 and 4B0

15 are the necessary and sufficient condition for an exactpenalty representation.

Lemma 3. Suppose that the assumptions of Theorem 1 hold and that there exists 4v̄1 r̄5 ∈ V ×R++ such that(42) holds. If, for all x ∈ Rn and r ≥ r̄ , f̄ 4x1 ·5 and �4·1 v̄1 r5 are �-lower semicontinuous at u = 0. Then (43)holds for all r > r̄ .

Proof. Let 4v̄1 r̄5 ∈ V ×R++ satisfy (42). If �f̄ 405= −�, then

arg minx∈Rn

�4x5= arg minx∈Rn

l4x1 v̄1 r5= �1 ∀ r ≥ r̄ 0

Now we assume that �f̄ 405 >−�. From (42) and the weak duality property, it is easy to show that, for all r ≥ r̄ ,

arg minx∈Rn

�4x5⊆ arg minx∈Rn

l4x1 v̄1 r50 (44)

Let r > r̄ . We now show that the reverse inclusion relation of (44) holds. Let x∗4r5 ∈ arg minx∈Rn l4x1 v̄1 r5.It follows from (42) that

�f̄ 405 = minx∈Rn

l4x1 v̄1 r5

= l4x∗4r51 v̄1 r5

= infu∈U4r5

8f̄ 4x∗4r51 u5+�4u1 v̄1 r590 (45)

Choose uk ∈U4r5 such that

limk→+�

8f̄ 4x∗4r51 uk5+�4uk1 v̄1 r59= infu∈U4r5

8f̄ 4x∗4r51 u5+�4u1 v̄1 r590 (46)

The above equation, together with (45), (46), and U4r5⊆U4r̄5, yields that

�f̄ 405 = limk→+�

8f̄ 4x∗4r51 uk5+�4uk1 v̄1 r59

= limk→+�

8f̄ 4x∗4r51 uk5+�4uk1 v̄1 r̄5+ãr̄4uk1 v̄1 r59

≥ lim supk→+�

{

infu∈U4r̄5

8f̄ 4x∗4r51 u5+�4u1 v̄1 r̄59+ãr̄4uk1 v̄1 r5

}

≥ lim supk→+�

8�̄4v̄1 r̄5+ãr̄4uk1 v̄1 r59

= �̄4v̄1 r̄5+ lim supk→+�

ãr̄4uk1 v̄1 r5

= �f̄ 405+ lim supk→+�

ãr̄4uk1 v̄1 r50

Dow

nloa

ded

from

info

rms.

org

by [

158.

132.

175.

28]

on 1

5 Se

ptem

ber

2015

, at 0

5:12

. Fo

r pe

rson

al u

se o

nly,

all

righ

ts r

eser

ved.


Noting that ãr̄4uk1 v̄1 r5≥ 0 ∀k, it follows from the relations above that

limk→+�

ãr̄4uk1 v̄1 r5= 00

From (A3), we obtain thatlim

k→+��4uk5= 00 (47)

By the fact that f̄ 4x∗4r51 ·5 and �4·1 v̄1 r5 are �-lower semicontinuous at u = 0, it follows from (45)–(47) andcondition (A1) that

infx∈Rn

�4x5 = limk→+�

8f̄ 4x∗4r51 uk5+�4uk1 v̄1 r59

≥ lim inf�4uk5→0

f̄ 4x∗4r51 uk5+ lim inf�4uk5→0

�4uk1 v̄1 r5

≥ f̄ 4x∗4r5105= �4x∗4r550

That is, x∗4r5 ∈ arg minx∈Rn �4x5. Thus (43) holds. �Theorem 3. Suppose that the assumptions of Theorem 1 hold. Then the following two statements hold.(i) if v̄ ∈ V supports an exact penalty representation of 4P5, then there exists r̄ ∈ R++ such that conditions

4B015 and 4B0

25 hold;(ii) assume that there exists 4v̄1 r̄5 ∈ V × R++ such that, for all x ∈ Rn and r ≥ r̄ , f̄ 4x1 ·5 and �4·1 v̄1 r5

are �-lower semicontinuous at u = 0. If conditions 4B015 and 4B0

25 hold, then v̄ supports an exact penaltyrepresentation of 4P5.

Proof. (i) Let v̄ support an exact penalty representation of 4P5. Then there exists r̄ ∈ R++ such that (42)holds, that is,

�f̄ 405= �̄4v̄1 r51 ∀ r ≥ r̄ 0

Thus, for all r ≥ r̄ 1 we have�f̄ 405≤ �f̄ 4u5+�4u1 v̄1 r51 ∀u ∈U4r50 (48)

In particular, when r = r̄ , (48) implies that condition 4B025 holds for W �

� 405∩U4r̄5. From (48) and condition (A1),we obtain that 0 ∈N4v̄1 r1 �5, for all �> �̄4v̄1 r5. Thus, for all r ≥ r̄ , it follows from (A1), (A2), and (A3) that

inf8ãr̄4u1 v̄1 r5 � u ∈N4v̄1 r1 �59= 01 ∀�> �̄4v̄1 r50

Then condition 4B015 holds.

(ii) Now we assume that conditions 4B015 and 4B0

25 hold. Let r > r̄ , �k ↓ �̄4v̄1 r5 (as k → +�5. From condi-tion 4B0

15, there exists 8uk9 satisfying

uk ∈N4v̄1 r1 �k51 (49)

limk→+�

ãr̄4uk1 v̄1 r5= 00 (50)

From (50) and condition (A3), we obtain that

limk→+�

�4uk5= 00 (51)

Thus, from (51) and U4r5⊆U4r̄5, when k is sufficiently large, uk ∈W �� 405∩U4r̄5. As such, it follows from 4B0

25,(49), and the monotonic nondecreasing property of � on r > 0 that

�k ≥ �f̄ 4uk5+�4uk1 v̄1 r5

≥ �f̄ 4uk5+�4uk1 v̄1 r̄5

≥ �f̄ 4050

Then�̄4v̄1 r5= lim

k→+��k ≥ �f̄ 4050

So combining with the weak duality property, we obtain (42). Again, by the assumptions of (ii), (42) andLemma 3, we see that (43) holds. �

Dow

nloa

ded

from

info

rms.

org

by [

158.

132.

175.

28]

on 1

5 Se

ptem

ber

2015

, at 0

5:12

. Fo

r pe

rson

al u

se o

nly,

all

righ

ts r

eser

ved.


From Theorem 3 and under the assumptions (A1), (A2), (A3), and (A4), we can obtain a necessary andsufficient condition for an exact penalty representation of (P).

Corollary 4. Assume that � satisfies conditions 4A15, 4A25, 4A35, and 4A45 and that there exists 4v̄1 r̄5 ∈

V ×R++ such that�̄4v̄1 r̄5 >−�0

If furthermore, for all x ∈ Rn and r ≥ r̄ , f̄ 4x1 ·5 and �4·1 v̄1 r5 are �-lower semicontinuous at u = 0, then v̄supports an exact penalty representation of 4P5 if and only if condition 4B0

25 holds.

Proof. It is obvious that the necessity can be obtained from Theorem 3(i). So we only need to prove thesufficiency. Without loss of generality, assume that 4B0

25 holds for r̄ and a neighborhood W �� 405 of 0 ∈ Rm1 .

That is,�f̄ 405≤ �f̄ 4u5+�4u1 v̄1 r̄51 ∀u ∈W �

� 405∩U4r̄50 (52)

For � > 0, by Lemma 1, there exists r� ≥ r̄ such that

N4v̄1 r�1 �5⊆W �� 405∩U4r�51 ∀� ∈ 4�̄4v̄1 r�51 �̄71 (53)

where �̄ >MP . Noticing that U4r�5⊆U4r̄5 and � is monotonically nondecreasing on r > 0, by (52), we have

�f̄ 405 ≤ �f̄ 4u5+�4u1 v̄1 r̄5

≤ �f̄ 4u5+�4u1 v̄1 r�51 ∀u ∈W �� 405∩U4r�50 (54)

Inequality (54) indicates that condition 4B025 holds for 4v̄1 r�5. Now let r ≥ r�0 It follows from �̄4v̄1 r5≥ �̄4v̄1 r�5,

the monotonically nondecreasing of � on r > 0 and (53) that

N4v̄1 r1 �5⊆N4v̄1 r�1 �5⊆W �� 405∩U4r�51 ∀� ∈ 4�̄4v̄1 r51 �̄70 (55)

Thus, for all u ∈N4v̄1 r1 �5 and � ∈ 4�̄4v̄1 r51 �̄7, from (54) and (55), we have that

�f̄ 405 ≤ �f̄ 4u5+�4u1 v̄1 r�5

≤ �f̄ 4u5+�4u1 v̄1 r5≤ �0

By the inequality above and (A1), we have 0 ∈N4v̄1 r1 �5 for any �> �̄4v̄1 r5. Thus, from (A1), (A2), and (A3),we get

inf8ãr�4u1 v̄1 r5 � u ∈N4v̄1 r1 �59= 01 ∀ r ≥ r�1 � > �̄4v̄1 r50

The inequality above indicates that condition 4B015 holds for 4v̄1 r�5. Again by the �-lower semicontinuity of

f̄ 4x1 ·5 and �4·1 v̄1 r5 at u= 0. From Theorem 3(ii), the conclusion can be obtained. �From Corollary 4, we have the following results.

Corollary 5. Assume that U4r5=Rm1 , � satisfies conditions (A1), (A3), and (A4) at �4u5= u and thereexists 4v̄1 r̄5 ∈ V ×R++ such that

�̄4v̄1 r̄5 >−�0

If, for all x ∈Rn and ∀ r ≥ r̄ , f̄ 4x1 ·5 and �4·1 v̄1 r5 are both lower semicontinuous at u= 0, then v̄ supports anexact penalty representation of (P) if and only if there exist r ′ ∈ R++ and a neighborhood W�405 of 0 ∈ Rm1

such that�f̄ 405≤ �f̄ 4u5+�4u1 v̄1 r ′51 ∀u ∈W�4050

Remark 5. In a Euclidean space, Burachik et al. [6, Theorem 3.3] can be derived from Corollary 5 byreplacing � with −�. It is worth noting that perturbation function � is assumed to be lower semicontinuous atu= 0 in Burachik et al. [6, Theorem 3.3], however, this hypothesis is not required in Corollary 5.

Remark 6. In a Euclidean space, Burachik and Rubinov [5, Theorem 6.2], which is a special case ofCorollary 5, missed a hypothesis, that is, the dualizing parameterization function f̄ 4x1 ·5 is lower semicontinuouson the whole space for all x ∈ X, because this hypothesis was used in the proof of Theorem 6.2 of Burachikand Rubinov [5]. Similarly, the special cases, Huang and Yang [11, Theorem 3.2] and Rubinov and Yang [20,Theorem 5.9], of Corollary 5 also missed this hypothesis. It is obvious that this hypothesis is stronger than thecorresponding hypothesis of Corollary 5, where it is only required that f̄ 4x1 ·5 is lower semicontinuous at u= 0.

Dow

nloa

ded

from

info

rms.

org

by [

158.

132.

175.

28]

on 1

5 Se

ptem

ber

2015

, at 0

5:12

. Fo

r pe

rson

al u

se o

nly,

all

righ

ts r

eser

ved.


In the study of an augmented Lagrangian dual problem in Nedich and Ozdaglar [13], conditions for the exis-tence of an exact penalty representation were not given. But in Nedich and Ozdaglar [13] the authors pointedout that it is significant to further study this problem (see Nedich and Ozdaglar [13, p. 605]). In the following,we will study the existence of an exact penalty representation in the framework of the augmented Lagrangiandual of 4P5.

The nonlinear augmenting penalty function in Nedich and Ozdaglar [13] is 41/r5�4ru5, see (12). We knowthat another nonlinear augmenting penalty function can be given as

�4u1 v1 r5= �u1 v� +1r�4ru51 4u1 v1 r5 ∈U4r5× 809×R++1

where 0 ∈ Rm1 , � is convex on an open set C ⊆ Rm1 with �405 = 0, and U4 · 5 is defined by (8). Thus, theexact penalty representation of 4P5 with �4u1 r5= 41/r5�4ru5 (that is, (42) and (43) hold with v̄ not appearingexplicitly) is equivalent to that v̄ = 0 supports an exact penalty representation. In this case,

N401 r1 �5=

{

u ∈U4r5

∣

∣

∣

∣

�f̄ 4u5+1r�4ru5≤ �

}

0

We will make our discussions in two cases as follows.Case 1. The function � is strictly convex on C.

From Example 1 (R1), we have that � satisfies (A1) and (A3) for �4u5= u. By the convexity of � , it followsthat �4·101 r5 is continuous on U4r5 for all r ∈R++. Therefore, a necessary and sufficient condition for an exactpenalty representation of 4P5 with this type of function �4u101 r5= 41/r5�4ru5 can be obtained by Theorem 3as follows.

Corollary 6. Let �4u101 r5 = 41/r5�4ru5, where 4u1 r5 ∈ U4r5×R++1� is a strictly convex function onC with �405 = 0 and C ⊆ Rm1 is an open convex set with 0 ∈ C, U4 · 5 is defined by (8). If f̄ 4x1 ·5 is lowersemicontinuous at u = 0, then v̄ = 0 supports an exact penalty representation of 4P5 if and only if there existr̄ ∈R++ and a neighborhood W�405 of 0 ∈Rm1 such that

(i) inf841/r5�4ru5− 41/r̄5�4r̄u5 � u ∈N401 r1 �59= 0, ∀ r ≥ r̄ 1 � > �̄401 r5;(ii) �f̄ 405≤ �f̄ 4u5+ 41/r̄5�4r̄u5, ∀u ∈W�405∩U4r̄5.

Case 2. The function � is nonnegative on C =Rm1 and satisfies conditions (p1) and (p2).From Example 2 (R4), we have that � satisfies (A1), (A2), and (A3) for �4u5= u+. By the nonnegativity of �

and �405= 0, it follows that �4·101 r5 is u+-lower continuous at 0 ∈Rm1 for all r ∈R++. Therefore, a necessaryand sufficient condition for an exact penalty representation of 4P5 with this type of functions can be obtainedby Theorem 3 as follows.

Corollary 7. Let �4u101 r5= 41/r5�4ru5, where 4u1 r5 ∈Rm1 ×R++, � is nonnegative on Rm1 and satisfiesthe following conditions:

(q1) � is convex on Rm1 and strictly convex on Rm1+ ;

(q2) for all 4u1 r̄5 ∈Rm1 ×R++ and r > r̄ , there holds

1r�4ru5−

1r̄�4r̄u5≥

1r�4ru+5−

1r̄�4r̄u+50

Then v̄ = 0 supports an exact penalty representation of 4P5 if and only if there exist r̄ ∈R++ and a neighborhoodW+

� 405 of 0 ∈Rm1 such that(i) inf841/r5�4ru5− 41/r̄5�4r̄u5 � u ∈N401 r1 �59= 0, ∀ r ≥ r̄ , �> �̄401 r5;

(ii) �f̄ 405≤ �f̄ 4u5+ 41/r̄5�4r̄u5, ∀u ∈W+� 405.

Similarly, no condition for the exact penalty representation was studied in Nedich and Ozdaglar [14]. Nowwe present such a condition for 4P5 using Corollary 4. The augmenting penalty function in Nedich andOzdaglar [14] is

�4u1 v1 r5= �u1 v� + r�4u51 4u1 v1 r5 ∈Rm1 ×Rm1 ×R++0

This is a linear function in r . Assume that the augmenting function � is nonnegative and convex on Rm1

with �405 = 0, and � satisfies the weak peak at zero condition. It is easy to show that � satisfies (A1), (A2),(A3), and (A4) for �4u5 = u+. If v̄ ≤ 0, by the definition of � and the property of � , �4·1 v̄1 r5 is u+-lower

Dow

nloa

ded

from

info

rms.

org

by [

158.

132.

175.

28]

on 1

5 Se

ptem

ber

2015

, at 0

5:12

. Fo

r pe

rson

al u

se o

nly,

all

righ

ts r

eser

ved.


semicontinuous, for all r ∈R++. So Corollary 4 implies a necessary and sufficient condition for v̄ to support anexact penalty representation of 4P5 as follows.

Corollary 8. Let �4u1 v1 r5 = �u1 v� + r�4u5, 4u1 v1 r5 ∈ Rm1 × Rm1 × R++, where � is nonnegative andconvex on Rm1 with �405= 0, and v̄ ≤ 0. If

(i) there exists r̄ ∈R++ such that �̄4v̄1 r̄5 >−�;(ii) the weak peak at zero condition holds; and

(iii) for any x ∈Rm1 , f̄ 4x1 ·5 is u+-lower semicontinuous at u= 0,then v̄ supports an exact penalty representation of 4P5 if and only if there exist r ′ ∈ R++ and a neighborhoodW+

� 405 of 0 ∈Rm1 such that�f̄ 405≤ �f̄ 4u5+ r ′�4u51 ∀u ∈W+

� 4050

Remark 7. If we remove the hypothesis of � being convex on Rm1 , then Corollary 8 still holds.

6. Conclusions. In this paper, we investigated an augmented Lagrangian dual framework for a primal prob-lem of minimizing an extended real-valued function, and obtained necessary and sufficient conditions for thezero duality gap and the exact penalty representation. We defined the nonlinear augmenting penalty functionon an open set and used a variable substitution for the dual variable. As such our framework includes somebarrier penalty functions and the weak peak at zero condition in the literature as special cases. We assumedneither a coercivity assumption nor a lower semicontinuous at zero assumption of the nonlinear augmentingpenalty function.

Acknowledgments. The authors would like to thank the two referees for their constructive comments andsuggestions on the early versions of the paper, which have significantly improved the presentation of the paper.The work in the present paper was partially supported by the Research Grants Council of the Hong KongSpecial Administrative Region, China, the National Natural Science Foundation of China [Grants 10831009,10971118, 10901096, 11271226, 11271233] and the Natural Science Foundation of Chongqing [Grant CSTC,2011BA0030].

References

[1] Balder EJ (1977) An extension of duality-stability relations to nonconvex optimization problems. SIAM J. Control Optim. 15:329–343.[2] Ben-Tal A, Zibulevsky M (1993) Penalty/Barrier Multiplier Methods: A new class of augmenting Lagrangian algorithms for large-scale

convex programming problems. Technical Report 4/93. Optimization Laboratory. Faculty of Industrial Engineering and Management.Technion-Israel Institute of Technology, Haifa, Israel, http://iew3.technion.ac.il/mcib/pbm0395.ps.gz.

[3] Bertsekas DP (1982) Constrained Optimization and Lagrange Multiplier Methods (Academic Press, New York).[4] Burachik RS, Kaya CY (2012) An augmented penalty function method with penalty parameter updates for non convex optimization.

Nonlinear Anal.: Theory, Methods Appl. 75:1158–1167.[5] Burachik RS, Rubinov A (2007) Abstract convexity and augmented Lagrangians. SIAM J. Optim. 18:413–436.[6] Burachik RS, Iusem AN, Melo JG (2010) Duality and exact penalization for general augmented Lagrangians. J. Optim. Theory Appl.

147(1):125–140.[7] Burachik RS, Iusem AN, Melo JG (2012) An inexact modified subgradient algorithm for primal–dual problems via augmented

Lagrangians. J. Optim. Theory Appl. 157(1):108–131.[8] Golshtein EG, Tretyakov NV (1996) Modified Lagrangians and Monotone Maps in Optimization (John Wiley & Sons, New York).[9] Gonzaga CC, Castillo RA (2003) A nonlinear programming algorithm based on non-coercive penalty functions. Math. Program.

96:87–101.[10] Hestenes MR (1969) Multiplier and gradient methods. J. Optim. Theory Appl. 4(5):303–320.[11] Huang XX, Yang XQ (2003) A unified augmented Lagrangian approach to duality and exact penalization. Math. Oper. Res. 28(3):

533–552.[12] Luo HZ, Sun XL, Li D (2007) On the convergence of augmented Lagrangian methods for constrained global optimization. SIAM J.

Optim. 18:1209–1230.[13] Nedich A, Ozdaglar A (2008) Separation of nonconvex sets with general augmenting functions. Math. Oper. Res. 33(3):587–605.[14] Nedich A, Ozdaglar A (2008) A geometric framework for nonconvex optimization duality using augmented Lagrangian functions.

J. Global Optim. 40:545–573.[15] Polyak R (1992) Modified barrier functions (theory and methods). Math. Program. 54:177–222.[16] Powell MJD (1968) A method for nonlinear constraints in minimization problems. 1969 Optim. Sympos., Univ. Keele, Keele (Academic

Press, London), 283–298.[17] Rockafellar RT (1993) Lagrange multipliers and optimality. SIAM Rev. 35:183–238.[18] Rockafellar RT, Wets RJ-B (1998) Variational Analysis (Springer-Verlag, Berlin).[19] Rubinov AM (2000) Abstract Convexity and Global Optimization (Kluwer Academic, Dordrecht, The Netherlands).[20] Rubinov AM, Yang XQ (2003) Lagrange-Type Functions in Constrained Non-Convex Optimization (Kluwer Academic Publisher,

Dordrecht, The Netherlands).[21] Rubinov AM, Huang XX, Yang XQ (2002) The zero duality gap property and lower semicontinuity of the perturbation function. Math.

Oper. Res. 27(4):775–791.

Dow

nloa

ded

from

info

rms.

org

by [

158.

132.

175.

28]

on 1

5 Se

ptem

ber

2015

, at 0

5:12

. Fo

r pe

rson

al u

se o

nly,

all

righ

ts r

eser

ved.


[22] Sun XL, Li D, McKinnon KIM (2005) On saddle points of augmented Lagrangians for constrained nonconvex optimization. SIAM J.Optim. 15:1128–1146.

[23] Tseng P, Bertsekas DP (1993) On the convergence of the exponential multiplier method for convex programming. Math. Program.60:1–19.

[24] Wang CY, Li D (2009) Unified theory of augmented Lagrangian methods for constrained global optimization. J. Global Optim.44:433–458.

[25] Wang CY, Zhou JC, Xu XH (2009) Saddle point theory of two classes of augmented Lagrangian and its applications to generalizedsemi-infinite programming. Applied Math. Optim. 59:413–434.

Dow

nloa

ded

from

info

rms.

org

by [

158.

132.

175.

28]

on 1

5 Se

ptem

ber

2015

, at 0

5:12

. Fo

r pe

rson

al u

se o

nly,

all

righ

ts r

eser

ved.

mathematics of operations research · mathematics of operations research 38(4), pp. 740–760,...

Documents