new results in program slicing

1

New Results inProgram Slicing

Aharon Abadi, Ran Ettinger, and Yishai Feldman

IBM Haifa Research Lab

2

Context

• The Programmer’s Apprentice– The Plan Calculus

• Bogart

• Midas

• Sliding

• Painless– Paz– Aderet

3

Improving Slice Accuracy by Compression of Data and Control Flow Paths

Presented at ESEC/FSE 2009

4

Program Slicing

Program

x := expStart Slice x := exp

Slice

The same sequence of values

5

A:Z0A:Z0

Control-Flow Path Compression

go-to Bgo-to B

if-zero-go-to A

test X

Work in two stages:- Compute the ‘traditional’ slice

- Control dependences- Data Dependences

- Compute the necessary branches to prevent infeasible control paths

test X

if-zero-go-to A

. . .

L:test Y

if-zero-

. . .

go-to L

B:

. . .

B:

6

A:Z0

go-to B

if-zero-go-to A

test X

Limitations of previous approaches:- insert all the loop;- add branches not from the program; or- do not preserve behavior

This algorithm:- preserves behavior- yields a sub-program

- one version may turn conditional branches into unconditional ones (“rhetorization”)

B:go-to Bgo-to B

test X test X

. . .

L:test Y

if-zero-

. . .

go-to L

B:

. . .

if-zero-go-to A

A:Z0

Control-Flow Path Compression

7

Data-Flow Path Compression

The result is too large

The value of R7 does not depend on the loop

R7:=exp1Out: R0:=R7 + 1

Previous syntax-preserving algorithms insert the loop and the assignments inside it

Out: R0:= R7 + 1

Start:R2:=0

R7:=exp1

Loop: R2:=R2 + 1

compare R2, R9

if-not-less-go-to Out

use R7

Temp:=R7; spill R7 to memory

… ; code that uses

; all registers

R7:=Temp; restore R7

go-to Loop

Start:R2:=0 R7:=exp1Loop: R2:=R2 + 1 compare R2, R9 if-not-less-go-to Out Temp:=R7 R7:=Temp go-to Loop Out: R0:=R7 + 1

8

Control-Flow Path Compressionx<11

F T

x:=x+1 goto A4

goto A2

y<TT F

y:=y-1

goto A

print(x)

x<9T F

x:=x-1x:=x+2

goto A2

goto A3

if (x<11)

x := x+1

goto A2

A1: if (y<T)

y := y–1

goto A1

goto A2

goto A4

x := x-1

A4: if (x<9) goto A3

A3: x := x+2

A2: print(x)

9

Compute the ‘Traditional’ Slicex<11

F T

x:=x+1 goto A4

goto A2

y<TT F

y:=y-1

goto A

print(x)

x<9T F

x:=x-1x:=x+2

goto A2

goto A3

if (x<11)

x := x+1

goto A2

A1: if (y<T)

y := y–1

goto A1

goto A2

goto A4

x := x-1

A4: if (x<9) goto A3

A3: x := x+2

print(x)

A2: print(x)

x:=x+1

x:=x+2 x:=x-1

x<11

x<9

y<T

10

Completing Control Flow Paths:Main Lemma

• precisely identifies the possible sets of branches that may be added to the slice

• any path in the original program can be chosen

• optimizations can be performed

All paths from the same point in the slice enter the slice at a single point

11

Compute the Necessary Branchesx<11

F T

x:=x+1 goto A4

goto A2

y<TT F

y:=y-1

goto A

print(x)

x<9T F

x:=x-1x:=x+2

goto A2

goto A3

if(x<11)

x:=x+1

goto A2

A1: if(y<T)

y:=y–1

goto A1

goto A2

goto A4

x:=x-1

A4: if(x<9) goto A3

A3: x:=x+2

A2: print(x)

12

Start:R2:=0 R7:=exp1Loop: R2:=R2 + 1 compare R2, R9 if-not-less-go-to Out use R7 Temp:=R7; spill R7 to ;memory … ; code that uses ;all registers R7:=Temp; restore R7 go-to Loop Out: R0:=R7 + 1

Data-Flow Path Compression

R7:=exp1Out:R0:=R7 + 1 +1

R7:=exp1

exit

R0:=R7+1

R2:=0

R2:=R2+1

compare R2,R9

if-not-less

use R7

Temp:=R7

R7:=Temp

goto Loop

go-to Out

13

++

++

exp1

Data-Flow Path Compression

R7:=exp1

exit

R7:=R7+1

R2:=0

R2:=R2+1

compare R2,R9

if-not-less

use R7

Temp:=R7

R7:=Temp

goto-Loop

• R7,Temp carry the value of exp1

• Use data edges instead of variables

go-to Out

out data portholds the last valuein data port

holds the next value

d1 d2

d1

Start:R2:=0 R7:=exp1Loop: R2:=R2 + 1 compare R2, R9 if-not-less-go-to Out use R7 Temp:=R7; spill R7 to ; memory … ; code that uses ; all registers R7:=Temp; restore R7 go-to Loop Out: R0:=R7+1

0

• The Plan Calculus:The Programmer’s Apprentice,Rich and Waters, 1990

14

exp1

entry

0

exit

++

R7

R0

R9

R2

++

R2

T F

compare R2,R9

R7:= exp1R0:=R7 + 1

Start:R2:=0

Loop: R2:=R2 + 1 compare R2, R9 if-not-less-go-to Out use R7 Temp:=R7; spill R7 to ; memory … ; code that uses ; all registers R7:=Temp; restore R7 go-to Loop Out: R0:=R7 + 1

R7:=exp1

Out: R0:=R7 + 1

R7:=exp1

if-not-less

use R7

15

exp1

0

exit

++

R7

R0

R9

R2

++

R2

T F

compare R2,R9

Start:R2:=0

Loop:R2:=R2 + 1 compare R2, R9 if-not-less- use R7 ; spill R7 to ; memory … ; code that uses ; all registers ; restore R7 go-to Loop Out: R0:=R7 + 1

R7:=exp1

if-not-less

use R7

Decompression

go-to Out

Temp:=R7

R7:=Temp

R7:=exp1

R0:=R7 + 1

go-to Out

entry

Out:

16

Properties of the Slices

• Syntax preserving, possibly rhetorizing• Behavior preserving• Executable• For structured programs

– At least as accurate as previous algorithms– Strictly smaller in interesting cases

• For unstructured programs– Empirically shown to be superior– Modification of the algorithm guaranteed at least as

accurate

17

Implementation

• A family of slicing algorithms– rhetorizing (*RB, *RM)– strictly syntax-preserving

(*PB, *PM)– amorphous (*AB, *AM)

• adds new branches(not from the program)

A1:if(y<T) goto A2

A:Z 0

if-zero-go-to A

test X

. . .

L:test Y

if-zero-go-to B

. . .

go-to L

C:

go-to exit

. . .goto exit

B:go-to C

18

Empirical Study

• Corpus of 15 manually-written assembly-language modules from a large mainframe product

• 8578 non-comment source lines

• Computed slices from all lines

• 5801 non-empty slices

19

Empirical Results

Effect of%slices better

%average decrease

%slices worse

%average decrease

Rhetorization177.5

Control path compression

Lenient BH3017

Strict BH9465

Data path compression

implemented124815

modified

20

Related WorkBehaviorPreserve

behaviorMay add infinite loops

Not executable

BH,CF1,Ag, HLB,*P,*R, *A

HLB, HDKH

Subset of the original program(for flat languages)

Syntax-preserving

RhetorizingAmorphous

BH, CF1, Ag, HD, HLB, *P

*RHLB, CF, *A

Comparison to traditional algorithm on structured programs

Smaller than traditional

Equal to traditional

Larger than traditional

*P, *R, *ABH, CF1, Ag, HD, KH, HLB, CF2

BH: Ball & Horwitz 1993CF: Choi & Ferrante 1994Ag: Agrawal 1994

KH: Kumar & Horwitz 2002HD: Harman & Danicic 1998HLB: Harman, Lakhotia & Binkley 2006

21

Conclusions

• Two techniques for reducing slice size– Control-Flow Path Compression

• Precise identification of all correct solutions• Shortest paths significantly improve slice accuracy

– 17-22% improvement for 30-37% of the cases– Data-Flow Path Compression

• Eliminates copy assignments• Yields significant improvement in a few cases

– 24% improvement for 1% of the slices computed

• Strictly smaller even for structured programs

22

Fine Slicing forProgram Transformation

23

Refactoring’s Rubicon:Extract Method

• Automating Extract Method is Refactoring’s Rubicon (Fowler*)– The one that demonstrates “serious tool

support”– Precondition for many other transformations

• This Rubicon has not yet been crossed– Getting it right requires more analysis

capability than is available in current IDEs

*http://www.martinfowler.com/articles/refactoringRubicon.html

24

Fowler’s Example (website)void printOwing() { printBanner();

//print details System.out.println("name: " + _name); System.out.println("amount " + getOutstanding());}

void printOwing() { printBanner(); printDetails(getOutstanding());}

void printDetails(double outstanding) { System.out.println("name: " + _name); System.out.println("amount " + outstanding);}

25

A Case Study inEnterprise Refactoring

• Converted a Java Servlet to use the MVC pattern*

• Used as much automated support as available– The whole conversion could be described as a series

of cataloged (“small”) refactorings– Most steps were inadequately supported by the IDE– Some were not supported at all

* Based on Alex Chaffee’s “Refactoring to Model-View-Controller” article (http://www.purpletech.com/articles/mvc/refactoring-to-mvc.html)

26

Case-Study: Automation (1)

13Total

3

3

2

1

1

1

1

1

Extract Method

Extract Temp

(Self) Encapsulate Field

Replace Magic Number with Symbolic Constant

Inline Temp

Extract Superclass

Delete Methods

Move Method

UsesFully Supported Refactorings

27

Case-Study: Automation (2)

23Total

10

5

3

2

1

1

1

Extract Method *

Substitute Expression **

Replace Temp with Query *

Replace Method with Method Object **

Substitute Statement **

Extract Class *

Move Statement (or Swap Statements) **

UsesPartial(*) or No(**) Support

28

Currently Unsupported Casesof Extract Method

(a) Extract multiple fragments

(b) Extract a partial fragment– select sub-expressions as parameters

(c) Extract loop with partial body– loop duplication with data flow

(d) Extract code with conditional exits

Program slicing pulls related code together!

29

slice (v.): to cut with or as if with a knife

Merriam-Webster

slice (n.): a thin flat piece cut from something

30

A (backward) slice of a given program with respect to selected “interesting” variables is a subprogram that computes the same values as the original program for the selected variables

A (backward) fine slice of a given program with respect to selected “interesting” variables and other “oracle” variables is a subprogram that computes the same values as the original program for the selected variables, given values for the oracle variables

31

Fine Slicing

• A generalization of traditional program slicing• Fine slices can be precisely bounded

– Slicing criteria include set of data and control dependences to ignore

• Fine slices are executable and extractable• Complement slices (co-slices) are also fine slices• Oracle-based semantics for fine slices• Algorithm for computing data-structure representing the

oracle• Forward fine slices are executable, may be slightly larger

than traditional forward slices• Confines generalize blocks for unstructured programs

32

Extract Computation

• A new refactoring

• Extracts a fine slice into contiguous code

• Computes the co-slice

• Computation can then be extracted into a separate method using Extract Method

• Passes necessary “oracle” variables between slice and co-slice

• Generates new containers if series of values need to be passed

33

(a) Extract multiple fragmentsUser user = getCurrentUser(request);

if (user == null) {

response.sendRedirect(LOGIN_PAGE_URL);

return;

}

response.setContentType("text/html");

disableCache(response);

String albumName = request.getParameter("album");

PrintWriter out = response.getWriter();

34

(b) Extract a partial fragment

out.println(DOCTYPE_HTML);

out.println("<html>");

out.println("<head>");

out.println("<title>Error</title>");

out.println("</head>");

out.print("<body><p class='error'>");

out.print("Could not load album '" +

albumName + "'");

out.println("</p></body>");

out.println("</html>");

35

out.println("<table border=0>");

int start = page * 20;

int end = start + 20;

end = Math.min(end,

album.getPictures().size());

for (int i = start; i < end; i++) {

Picture picture = album.getPicture(i);

printPicture(out, picture);

}

out.println("</table>");

(c) Extract loop with partial body

1

2

3

4

5

6

7

8

9

10

36

2

3

4

5

***

***

6

7

***

9

1

6

8

10

int start = page * 20;

int end = start + 20;

end = Math.min(end,

album.getPictures().size());

Queue<Picture> pictures =

new LinkedList<Picture>();

for (int i = start; i < end; i++) {

Picture picture = album.getPicture(i);

pictures.add(picture);

}

out.println("<table border=0>");

for (int i = start; i < end; i++)

printPicture(out, pictures.remove());

out.println("</table>");

37

(d) Extract code with conditional exits

if (album == null) {

new ErrorPage("Could not load album '"

+ album.getName() + "'").printMessage(out);

return;

}

//...

38

if (invalidAlbum(album, out))

return;

}

//...

boolean invalidAlbum(Album album,

PrintWriter out) {

boolean invalid = album == null;

if (invalid) {

new ErrorPage("Could not load album '"

+ album.getName() + "'").printMessage(out);

}

return invalid;

}

39

++

out.println("<table border=0>");int start = page * 20;int end = start + 20;end = Math.min(end, album.getPictures().size());for (int i = start; i < end; i++) { Picture picture = album.getPicture(i); printPicture(out, picture);}out.println("</table>");

entry

println

out

*

album

getPictures

size

page

min

+ out

start

end

T F

>

getPicture

i

out

end

printPicture

out

out

println

i

"<table border=0>"

20

"</table>"

exit

p1

p1

p2

p2

Token Semantics

40

++

out.println("<table border=0>");int start = page * 20;int end = start + 20;end = Math.min(end, album.getPictures().size());for (int i = start; i < end; i++) { Picture picture = album.getPicture(i); printPicture(out, picture);}out.println("</table>");

entry

println

out

*

album

getPictures

size

page

min

+ out

start

end

T F

>

getPicture

i

out

end

printPicture

out

out

println

i

"<table border=0>"

20

"</table>"

exit

printPicture

Fine Slicing

41

++

out.println("<table border=0>");for (int i = start; i < end; i++) { printPicture(out, picture);}out.println("</table>");

entry

println

out

out

T F

>

i

out

end

printPicture

out

out

println

i

"<table border=0>"

"</table>"

exit

printPicture

startpicture

The Fine Slice

42

++

out.println("<table border=0>");int start = page * 20;int end = start + 20;end = Math.min(end, album.getPictures().size());for (int i = start; i < end; i++) { Picture picture = album.getPicture(i); printPicture(out, picture);}out.println("</table>");

entry

println

out

*

album

getPictures

size

page

min

+ out

start

end

T F

>

getPicture

i

out

end

printPicture

out

out

println

i

"<table border=0>"

20

"</table>"

exit

printPicture

Co-Slicing

43

++

int start = page * 20;int end = start + 20;end = Math.min(end, album.getPictures().size());for (int i = start; i < end; i++) { Picture picture = album.getPicture(i); }

entry

*

album

getPictures

size

page

min

+

start

end

T F

>

getPicture

i

end

out

i

20

exit

startpicture

The Co-Slice

44

++

entry

*

album

getPictures

size

page

min

+

start

end

T F

>

getPicture

i

end

out

i

20

exit

start

picture

++

entry

println

out

T F

>

end

out

println

i

"<table border=0>"

"</table>"

exit

printPicture

startpicture

Fine slice Co-slice

out

45

++

println

>

remove

printPicture println

++

out.println("<table border=0>");int start = page * 20;int end = start + 20;end = Math.min(end, album.getPictures().size());Queue<Picture> pictures = new LinkedList<Picture>();for (int i = start; i < end; i++) { Picture picture = album.getPicture(i); pictures.add(picture); printPicture(out,pictures.remove());}out.println("</table>");

entry

println

out

*

album

getPictures

size

page

min

+ out

start

end

T F

>

getPicture

i

out

end

printPicture

out

out

println

i

"<table border=0>"

20

"</table>"

exit

new

remove

add

picture

pictures

picture

pictures

pictures

Adding a Container

pictures

46

++

println

<

remove

printPicture println

++

void display(PrintStream out, int start, int end, Queue<Picture> pictures){ out.println("<table border=0>"); for (int i = start; i < end; i++) { printPicture(out, pictures.remove()); } out.println("</table>");}

entry

println

out

out

start

T F

>

out

end

printPicture

out

println

i

"<table border=0>"

"</table>"

exit

pictures

remove

entry

i

out

The Fine Slice

pictures

pictures

picture

47

++

println

>

remove

printPicture println

++

entry

println

out

*

album

getPictures

size

page

min

+ out

start

end

T F

>

getPicture

i

out

end

printPicture

out

out

println

i

"<table border=0>"

20

"</table>"

exit

new

remove

add

out.println("<table border=0>");int start = page * 20;int end = start + 20;end = Math.min(end, album.getPictures().size());Queue<Picture> pictures = new LinkedList<Picture>();for (int i = start; i < end; i++) { Picture picture = album.getPicture(i); pictures.add(picture); printPicture(out,pictures.remove());}out.println("</table>");

Program with

Container

pictures

pictures

pictures

pictures

picture

picture

48

++

>

++

int start = page * 20;int end = start + 20;end = Math.min(end, album.getPictures().size());Queue<Picture> pictures = new LinkedList<Picture>();for (int i = start; i < end; i++) { Picture picture = album.getPicture(i); pictures.add(picture); }display(out, start, end, pictures);

entry

*

album

getPictures

size

page

min

+

out

start

end

T F

>

getPicture

i

end

i

20

exit

newpictures

add

display

pictures

start

out

The Co-Slice

pictures

pictures

pictures

picture

49

Conclusions

• Fine slicing algorithm yields executable slices whose boundaries can be precisely controlled

• Can be used to make any subset of a program executable by adding some control structures but not the data on which they depend– including forward slices, thin slices, barrier

slices, chops, and barrier chops– Conjecture: the size of these executable

programs will not be substantially larger

50

Conclusions

• New Extract Computation refactoring is an important step towards the automation of Extract Method in difficult cases– Enables the automation of big refactorings

from smaller building blocks

• Uses new fine-slicing algorithm• Automatically computes complement

slices (co-slices)• Automatically generates containers to

pass series of values if necessary

51

Related Work (I): Non-Executable Slices

• Traditional backward slicing (e.g., Weiser [ICSE81] or Ottenstein & Ottenstein [PSDE84]), when applied to unstructured code– Solved by path-completion stage in plan-based slicing (Abadi,

Ettinger & Feldman [FSE09])

• Forward slicing (Horwitz, Reps & Binkley, [TOPLAS90])• Barrier slicing (Krinke [SCAM03])• Chopping (Jackson & Rollins [FSE94]) and Barrier

Chopping (Krinke [SCAM03])• Thin slicing (Sridharan, Fink & Bodik [PLDI07])• All the above can be made executable with an

appropriate oracle, by adding the required control structure

52

Related Work (II): Executable Slices with Reduced Scope or Size

• Block-based slicing (Maruyama [SSR01]): structured code only, no correctness proof

• Co-slicing (Ettinger's thesis, Oxford 2006): limited to slicing from the end and oracle of final values only; proof on toy language

• Parametric slicing (Field, Ramalingam & Tip [POPL95]): an executable generalization of static and dynamic slices; like oracle semantics, they formalize programs with holes; however, their holes stand for expressions whose values are irrelevant, while our holes stand for significant (oracle) values

• Some forms of dynamic and forward slicing are executable (Binkley et al. [SCAM04]): forward slices made excessively large through the addition of backward slices

53

Related Work (III): Behavior- Preserving Procedure Extraction

• Contiguous code– Bill Opdyke's thesis (UIUC 1992): for C++– Griswold and Notkin [ToSE93]: for Scheme

• Arbitrary selections– Tucking (Lakhotia & Deprez [IST98]): the complement is a slice too; no dataflow from the

extracted slice to its complement yields over-duplication; strong preconditions (e.g., no global variables involved, and no live-on-exit variable defined in both the slice and complement)

– Semantics-Preserving Procedure Extraction (Komondoor & Horwitz [POPL00]): considers all permutations of selected and surrounding statements; no duplication allowed; not practical (exponential time complexity); very strong preconditions

– Effective Automatic Procedure Extraction (Komondoor & Horwitz [IWPC03]): improves on their previous algorithm by improving complexity (cubic time and space), allowing some duplication (of conditionals and jumps); might miss some correct permutations; no duplication of assignments or loops; allows dataflow from complement to extracted code and from extracted code to (the second portion of the) complement; supports extraction of returns

– Extraction of block-based slices (Maruyama [SSR01]): extracts a slice of one variable only; restricted to structured code; no proof given

– Ettinger's thesis (Oxford 2006): sliding transformation sequentially composes a slice and its complement, allowing dataflow from the former to the latter; supports loop untangling and duplication of assignments; restricted to slicing from the end, and only final values from the extracted slice can be reused in the complement; proof for toy language