spreadsheets for developers

142
Spreadsheets for developers Felienne Hermans @Felienn e

Upload: felienne-hermans

Post on 03-Nov-2014

24 views

Category:

Engineering


4 download

DESCRIPTION

Spreadsheets are often dismissed by developers for not being "proper programming" but that is not true. Since I have shown that spreadsheets are Turing complete, you have no excuse to diss them any longer. In this session, I will implement various algorithms in Excel to show you its power and elegance. After all, spreadsheets are 'live' and functional, so they have everything going for them! Furthermore they are very fit for TDD and rapid prototyping. Don't fight spreadsheets any longer, but learn to love them.

TRANSCRIPT

Page 1: Spreadsheets for developers

Spreadsheets for developersFelienne Hermans

@Felien

ne

Page 2: Spreadsheets for developers

So you are a developer? Why should you bother to learn spreadsheets?

In this deck, I explain you why.

Page 3: Spreadsheets for developers

So you are a developer? Why should you bother to learn spreadsheets?

In this deck, I explain you why.

People often think about spreadsheets as data, but that is a gross misslabeling.

Page 4: Spreadsheets for developers

Spreadsheets are code

Page 5: Spreadsheets for developers

Spreadsheets are code

I have made it my life’s work to spread the happy word

“Spreadsheets are code!”

Page 6: Spreadsheets for developers

Spreadsheets are code

I have made it my life’s work to spread the happy word

“Spreadsheets are code!”

If you don’t immediately believe me, I have three reasons*

* If you do believe me, skip the next 10 slides ;)

Page 7: Spreadsheets for developers

Spreadsheets are code

Page 8: Spreadsheets for developers

1) Used for similar problems

Page 9: Spreadsheets for developers

This tool (for stock price computation) could have been built in any language. C, JavaScript, COBOL, or Excel.

The problems Excel is used for are often (not always) similar to problems solved in different languages.

Page 10: Spreadsheets for developers

I go to great lengths to make my point. To such great lengths that I built a Turing machine in Excel, using formulas only.

Page 11: Spreadsheets for developers

Here you see it in action. Every row is an consecutive step of the tape.

This makes it, in addition to a proof that formulas are Turing complete,Also a nice visualization of a Turing machine.

Page 12: Spreadsheets for developers

2) Formulas are Turing complete

Here you see it in action. Every row is an consecutive step of the tape.

This makes it, in addition to a proof that formulas are Turing complete,Also a nice visualization of a Turing machine.

Many people liked it :)

Page 13: Spreadsheets for developers

3) They suffer from the same problems

Finally, spreadsheets suffer from typicial ‘software’ problems like lack of documentation and a long lifespan during which many different users are involved.

Page 14: Spreadsheets for developers

Spreadsheets are code

In summary: both the activities, complexity and problems are the

same

Page 15: Spreadsheets for developers

And not just a programminglanguage!

Page 16: Spreadsheets for developers

I argue that

Excel is the next

language

to learn

Resistance is futile!

Page 17: Spreadsheets for developers

Stukje Bret Victor hierlive programming

Spreadsheet are ‘live programming’ avant la lettre.

What Bret Victor is been advocating for lately, we had that since VisiCalc!

Just type up your formula and you will get the result immediately.

Page 18: Spreadsheets for developers

pure

functionalYou love pure functional languages?

Page 19: Spreadsheets for developers

pure

functionalYou love pure functional languages?

We’ve got you covered. All a formula can do is take input and do something with it.

No side effects possible.

Page 20: Spreadsheets for developers

Lingua

franca of

computing

Finally, spreadsheets are the lingua franca of computing. You grandfather probably knows how to put a spreadsheet together for his savings. Your next door neighbour? Uses it for his fantasy football league. That history major you are helping out with his math work? I am sure he out-pivot tables you!

Everyone knows this, expect for developers!

That’s just crazy!

Page 21: Spreadsheets for developers

Excel is the next

language

to learn

I am just going to assume you are convinced and you want to learn spreadsheets. Therefore, I’ll implement selection sort in a spreadsheet to show you their power.

Page 22: Spreadsheets for developers

Excel is the next

language

to learn

I am just going to assume you are convinced and you want to learn spreadsheets. Therefore, I’ll implement selection sort in a spreadsheet to show you their power.

To refresh your memory: selection sort sorts a list by repeatedly selecting the smallest value and putting it in the front:

Page 23: Spreadsheets for developers

Excel is the next

language

to learn

I am just going to assume you are convinced and you want to learn spreadsheets. Therefore, I’ll implement selection sort in a spreadsheet to show you their power.

To refresh your memory: selection sort sorts a list by repeatedly selecting the smallest value and putting it in the front:

7-8-3-1-12-15

Page 24: Spreadsheets for developers

Excel is the next

language

to learn

I am just going to assume you are convinced and you want to learn spreadsheets. Therefore, I’ll implement selection sort in a spreadsheet to show you their power.

To refresh your memory: selection sort sorts a list by repeatedly selecting the smallest value and putting it in the front:

7-8-3-1-12-15 |1 is the minimum, swap it with 7

Page 25: Spreadsheets for developers

Excel is the next

language

to learn

I am just going to assume you are convinced and you want to learn spreadsheets. Therefore, I’ll implement selection sort in a spreadsheet to show you their power.

To refresh your memory: selection sort sorts a list by repeatedly selecting the smallest value and putting it in the front:

1-8-3-7-12-15 |1 is the minimum, swap it with 7

Page 26: Spreadsheets for developers

Excel is the next

language

to learn

I am just going to assume you are convinced and you want to learn spreadsheets. Therefore, I’ll implement selection sort in a spreadsheet to show you their power.

To refresh your memory: selection sort sorts a list by repeatedly selecting the smallest value and putting it in the front:

1-8-3-7-12-15 |Now 3 is the minimum, swap it with 8

Page 27: Spreadsheets for developers

Excel is the next

language

to learn

I am just going to assume you are convinced and you want to learn spreadsheets. Therefore, I’ll implement selection sort in a spreadsheet to show you their power.

To refresh your memory: selection sort sorts a list by repeatedly selecting the smallest value and putting it in the front:

1-3-8-7-12-15 |Rinse and repeat

Page 28: Spreadsheets for developers

What we want to

sort

Page 29: Spreadsheets for developers

The index

Page 30: Spreadsheets for developers

The first formulas is obvious: find the

minimum

Page 31: Spreadsheets for developers

Little known fact: you

could also use 3:3 here,

to indicate the 3th row

(much like the more

known A:A for the A

column)

Page 32: Spreadsheets for developers

Next is finding

the location of

the minimum,

we can do that

with MATCH

MATCH takes as arguments: the

search value, the range to search in

and the type of match (exact or bigger/smaller)

Page 33: Spreadsheets for developers

MATCH results in 10:

the minimum is found on index

10

Page 34: Spreadsheets for developers

Let’s start small and make a formula that

places an “X" in the swap spots

Page 35: Spreadsheets for developers

So: if our index is equal to the index of the minimum,

we swap (X) otherwise we do

nothing (_)

Let’s start small and make a formula that

places an “X" in the swap spots

Page 36: Spreadsheets for developers

So: if our index is equal to the index of the minimum,

we swap (X) otherwise we do

nothing (_)

That looks cool, let’s drag it

right

Page 37: Spreadsheets for developers

Awww... Something went

wrong!

Page 38: Spreadsheets for developers

By default, Excel transforms formulas by

location, so this one is changed

incorrectly

Page 39: Spreadsheets for developers

If we do not want that, we

add a $ before a reference to fix it. Now, only the row is updated

Page 40: Spreadsheets for developers

If we do not want that, we

add a $ before a reference to fix it. Now, only the row is updated

Let’s try dragging again

Page 41: Spreadsheets for developers

It works!

Page 42: Spreadsheets for developers

Let’s also fix the index row here, because we are dragging all this

down later

Page 43: Spreadsheets for developers

We will swap based on the

index, starting at 1

Page 44: Spreadsheets for developers

We also the

swap value,

which we can

find with INDEX

Page 45: Spreadsheets for developers

We also the

swap value,

which we can

find with INDEX

INDEX takes as arguments: the

range to locate a value in, followed

by the row and column

Page 46: Spreadsheets for developers

INDEX takes as arguments: the

range to locate a value in, followed

by the row and column

We also the

swap value,

which we can

find with INDEXWe use row 1 (as we are looking in only one row) and the column in B4 (the index of the

swap)

Page 47: Spreadsheets for developers

With this, we can edit the

second branch of the if, to add the second swap

situation

Page 48: Spreadsheets for developers

With this, we can edit the

second branch of the if, to add the second swap

situation

We also swap is the index is equal to the ‘swap index’

Page 49: Spreadsheets for developers

Works! We marked both swap spots with

an X

Page 50: Spreadsheets for developers

Let’s fill in the easiest blank

first, the _In case we do not swap, we can just

use the value above

Page 51: Spreadsheets for developers

Let’s fill in the easiest blank

first, the _In case we do not swap, we can just

use the value above

Page 52: Spreadsheets for developers

Before we go any further, I added

conditional formatting to indicate the swap spots

Page 53: Spreadsheets for developers

So what goes on this spot? If the index is equal to the swap

spot...

Page 54: Spreadsheets for developers

So what goes on this spot? If the index is equal to the swap

spot, we output the minimum

Page 55: Spreadsheets for developers

And if the index is swap,

we output the swap

value

Page 56: Spreadsheets for developers

Swapped!

Page 57: Spreadsheets for developers

Swapped!

Looks like we are ready to draw all formulas

down

Page 58: Spreadsheets for developers
Page 59: Spreadsheets for developers

Whoops! Something went wrong. Can you spot what?

Page 60: Spreadsheets for developers

1 is picked as minimum

everywhere

Page 61: Spreadsheets for developers

1 is picked as minimum

everywhere

This range needs to

shift right every step

Page 62: Spreadsheets for developers

We can use the OFFSET function for

that

Page 63: Spreadsheets for developers

We can use the OFFSET function for

that

OFFSET takes as arguments: the

range you want to shift, followed by number of rows and number of

columns

Page 64: Spreadsheets for developers

We can use the OFFSET function for

that

OFFSET takes as arguments: the

range you want to shift, followed by number of rows and number of

columnsWe’ll shift no rows down and B4 (swapindex)-1 left

Page 65: Spreadsheets for developers

Works!

Page 66: Spreadsheets for developers

But let’s make

things a bit

scarier

Page 67: Spreadsheets for developers

PopQuiz!What does this mean?

Page 68: Spreadsheets for developers

PopQuiz!What does this mean?

Let me

give

you a hint

Page 69: Spreadsheets for developers

Let me

give

you a hint

= Range1 Range2 results in a

referece to the intersection of the two ranges. In this case C5 with value

3.

Page 70: Spreadsheets for developers

= Range1 Range2 results in a

referece to the intersection of the two ranges. In this case C5 with value

3.

Page 71: Spreadsheets for developers

We can use this to make our formula easier. Let’s call this

range Index

Page 72: Spreadsheets for developers

With that,

this

becomes...

Page 73: Spreadsheets for developers
Page 74: Spreadsheets for developers

And adding ‘ E:E’ is optional. If you remove it, Excel assumes you want the intersection of the range and the cell you are in.

Page 75: Spreadsheets for developers

And adding ‘ E:E’ is optional. If you remove it, Excel assumes you want the intersection of the range and the cell you are in.

So we

can

simplify

Page 76: Spreadsheets for developers

And adding ‘ E:E’ is optional. If you remove it, Excel assumes you want the intersection of the range and the cell you are in.

So we

can

simplify

Page 77: Spreadsheets for developers

We can

repeat

this trick

Page 78: Spreadsheets for developers

We can

repeat

this trick

So it

becomes...

Page 79: Spreadsheets for developers

Isn’t that nice?

Page 80: Spreadsheets for developers

Isn’t that nice?

Page 81: Spreadsheets for developers

There’s just

one

problem,

that ugly

E3!

Isn’t that nice?

Page 82: Spreadsheets for developers

To fix that, we

need to dive into

named ranges

A2:B7

Page 83: Spreadsheets for developers

So far, we

have used

named ranges,

to name, well,

ranges.

Page 84: Spreadsheets for developers

So far, we

have used

named ranges,

to name, well,

ranges.But we

can also

name:

Page 85: Spreadsheets for developers

So far, we

have used

named ranges,

to name, well,

ranges.But we

can also

name:strings

All You Need is

Love...

Page 86: Spreadsheets for developers

All You Need is

Love...

Page 87: Spreadsheets for developers

Love is All You

Need!

Page 88: Spreadsheets for developers

Love is All You

Need!

But we can name funkier

stuff, let’s stick with the love theme!

Page 89: Spreadsheets for developers

“However much I love

you,

You will always love

me more”

Page 90: Spreadsheets for developers

“However much I love

you,

You will always love

me more”

This too can be expressed with a named

range

Page 91: Spreadsheets for developers

We can put a

constant in

(nothing new

so far)

Page 92: Spreadsheets for developers
Page 93: Spreadsheets for developers

But we can also

put a formula in

Page 94: Spreadsheets for developers
Page 95: Spreadsheets for developers

Looks like a

range, but is a

formula

Page 96: Spreadsheets for developers

“Everyday I love you more”

Named ranges got you

covered again!

Page 97: Spreadsheets for developers

What we want now, is to

refer to the cell in C4, and then increase

its value

Page 98: Spreadsheets for developers

ROW(cell) results in the row of a cell,

for example ROW(A8) = 8

Without arguments ROW returns the

current row.

What we want now, is to

refer to the cell in C4, and then increase

its value

We can use the ROW for

that

Page 99: Spreadsheets for developers

ROW(cell) results in the row of a cell,

for example ROW(A8) = 8

Without arguments ROW returns the

current row.3 in this case

What we want now, is to

refer to the cell in C4, and then increase

its value

We can use the ROW for

that

Page 100: Spreadsheets for developers

We can use that to create the address of the cell above,

as such

Page 101: Spreadsheets for developers

The cell

above

Page 102: Spreadsheets for developers

But we need the value of

the cell rather than the address.

Excel’s got you covered!

Page 103: Spreadsheets for developers

But we need the value of

the cell rather than the address.

Excel’s got you covered!W

e can use

INDIRECT for

this

Page 104: Spreadsheets for developers

INDIRECT turns a string into a

reference, and is in that sense similar

to the ‘eval’ of JavaScript

But we need the value of

the cell rather than the address.

Excel’s got you covered!W

e can use

INDIRECT for

this

Page 105: Spreadsheets for developers

So this formula

Page 106: Spreadsheets for developers

So this formula

Is equal to this one

Page 107: Spreadsheets for developers

We can use INDIRECT to get the

reference to the above cell

Page 108: Spreadsheets for developers

The name is now equal to the value in

the cell above

Page 109: Spreadsheets for developers
Page 110: Spreadsheets for developers

Just add the ‘van Buuren factor’

Page 111: Spreadsheets for developers
Page 112: Spreadsheets for developers

Now we have a formula in a

named range that depends

on the cell you call it from

Page 113: Spreadsheets for developers

We can use this named

ranged trick to get rid of that

UGLY E3!

Page 114: Spreadsheets for developers

We can use this named

ranged trick to get rid of that

UGLY E3!By making a

named range

which points to

the row above

(we use the

row:row syntax)

Page 115: Spreadsheets for developers

TADAAA!

Page 116: Spreadsheets for developers

TADAAA!

Doesn’t that read like a

novel?For comparison, I

have written selection sort in

Python

Page 117: Spreadsheets for developers

Similar, but a lot

less concise! :)

Page 118: Spreadsheets for developers

If spreadsheets are code, can we apply software engineering methods to improve them?

Page 119: Spreadsheets for developers

If spreadsheets are code, can we apply software engineering methods to improve them?

That is the central

research question of

my dissertation

Page 120: Spreadsheets for developers

The conclusion is:

More info: felienne.com/archives/2

534

Page 121: Spreadsheets for developers

Because SE methods transfer so well, after my graduation, I built a spreadsheet refactoring tool called BumbleBee.

Page 122: Spreadsheets for developers

Because SE methods transfer so well, after my graduation, I built a spreadsheet refactoring tool called BumbleBee.

Here you see the user interface in Excel 2010.

Page 123: Spreadsheets for developers
Page 124: Spreadsheets for developers

This formula is ‘smelly’:

it can be improved by using an AVERAGE

Page 125: Spreadsheets for developers

You can ask BumbleBee for

rewrites to apply

Page 126: Spreadsheets for developers

BumbleBee

suggests a

refactoring

Page 127: Spreadsheets for developers

And shows you how the new

formula will look like

Page 128: Spreadsheets for developers

Click apply and your formula will be refactored!

Page 129: Spreadsheets for developers

Click apply and your formula will be refactored!

Page 130: Spreadsheets for developers

The transformations are programmable, with a small

language

Page 131: Spreadsheets for developers

And of course, if you say refactoring...

Page 132: Spreadsheets for developers

And of course, if you say refactoring, you say testing!

When users modify their spreadsheet, theywant to be sure the fuinctionality of their spreadsheets remains the same.

But how to get end-users to test?

This is already hard for professional developers!

Page 133: Spreadsheets for developers

And of course, if you say refactoring, you say testing!

When users modify their spreadsheet, theywant to be sure the fuinctionality of their spreadsheets remains the same.

But how to get end-users to test?

This is already hard for professional developers!

But spreadsheet users are already good testers!

Page 134: Spreadsheets for developers

Look at that!

It is like a test

Page 135: Spreadsheets for developers

Look at that!

It is like a testWe figured that rather

than learning spreadsheet users a new tool, we could exploit these formulas

Page 136: Spreadsheets for developers

So we built Expector: a tool that can detect these test formulas and save them in a ‘test suite’

Page 137: Spreadsheets for developers

Once the test

formulas are saved,

we can run them

and validate the

outcome

Page 138: Spreadsheets for developers

Once the test

formulas are saved,

we can run them

and validate the

outcomeWe can even show ‘coverage’, by taking the cell dependencies into account

Page 139: Spreadsheets for developers

We can even show ‘coverage’, by taking the cell dependencies into account

Page 140: Spreadsheets for developers

In a similar fashion, we can visualize non-tested cells, to help direct testing effort.

Page 141: Spreadsheets for developers

In a similar fashion, we can visualize non-tested cells, to help direct testing effort.

Also available:felienne.com/Expector

Page 142: Spreadsheets for developers

That’s all folks! Thanks for watching my talk on SlideShare!

Don’t forget that:

More info?• www.felienne.com• www.spreadsheetlab.org

Want to connect?• @felienne • [email protected]

Spreadsheets are code