analysis of algorithms

Richard Kelley

welcome! You’re in SEM 261. This is analysis of algorithms Please make sure you’re in the right place.

administrivia We have to get through some boring stuff

today. Class Policies

course webpages who I am textbooks homework quizzes & exams

we’ll also do a little bit of fun stuff. No heavy lifting today.

course webpage & syllabus The main course page is

http://www.cse.unr.edu/~rkelley/algorithms.html

You’ll find the following there: Syllabus Schedule Slides Example Programs

Possibly other sites, as demand dictates Code submission? Blog with lecture notes?

meet the course staff I’m Richard Kelley

call me Richard PhD Student – don’t call me “Dr” or

“Professor” Social Artificial Intelligence & Machine Learning Robotics Multiagent Systems

I took this class as an undergrad at UW. I’ve TA’ed for this class for a few years here.

textbooks no required text two (highly) recommended texts:

Algorithm Design by Kleinberg & Tardos Introduction to Algorithms by Cormen, et. al.

instead of required textbooks my slides slides from previous semesters links to online resources

videos short notes sample programs

all available from the course website.

when I use a resource, I will point you to it.

homework regularly assigned

I’m aiming for at least one per week some really easy

as in, “make sure you can compile this program.” some more involved

programming writing

English exposition proofs

first (easy) one today . . . maybe. Linux only

quizzes and exams necessary evil regular quizzes to make sure you’re paying

attention these should be easy, count as participation

a midterm about half way through the material

a final comprehensive

grading homework (55% of your grade)

some theory some programming some writing

midterm (15% of your grade) final (25% of your grade) attendance & participation (5% of your grade)

short quizzes

what’s this course about? we want to write good programs.

most advanced courses you take cover topics operating systems networking graphics robotics

some courses are about the processes that lead to better code human-computer interaction software engineering

the goal is that at the end of this course, your programs will be better regardless of topic regardless of process

what is a good program? I know it when I see it? some common qualities

easy to read easy to modify lots of tests correct! fast no memory leaks

what’s wrong with these measures? nothing! but we’re going to focus (mostly) on the last

two.

we want to write programs that are fast. we want to write programs that use little

memory.

we’re going to take a slightly more general view, though.

not programs. algorithms. it turns out that focusing on programs is a bad

idea. computers tend to get faster over time.

one way to make your program faster is to buy a better machine.

although faster machines are good, we can do better if we’re more careful in our design. this means choosing better algorithms

what is an algorithm? we can be really picky here

but we won’t be for our purposes, an algorithm is

a step-by-step process for a well-defined problem takes zero or more inputs produces zero or more outputs is finite is clearly defined

we’ll use pseudocode to describe algorithms

what is a good algorithm? most problems can be solved in many

different ways, using many different algorithms.

some algorithms are better than others faster or more space efficient

we want to choose the “best” algorithms let’s focus on time and try to work out what

this means. (I’m following Kleinberg & Tardos in this process)

why time? even though there are a few quantities we

want to optimize our program for, we’re going to spend most of our energy on running time.

this is pretty standard. many of the ideas here carry over to other

types of analysis.

basic notation we need to have notation for

the size of an input the running time of the algorithm

we’ll use variables like n and m to talk about input sizes. we focus on input sizes instead of inputs

running time often depends more on the size of the input than the actual value of the input.

we’ll use T( ) to talk about running times. for example, T(n) is the running time of an

algorithm on an input of size n.

attempt 1: good means fast idea: implement the algorithm. run it on some

inputs, record the times this is acceptable in some cases

in the real world, profiling our programs is a common optimization.

this approach has fundamental limitations that we want to (and can easily) avoid.

at least one homework assignment will involve this kind of analysis though.

why does this fail? problems:

sensitive to the computer we run on sensitive to the quality of our implementation sensitive to the inputs we choose doesn’t tell us how running time scales with

increasing problem sizes. scale:

if I run two different algorithms on inputs of size one, and then I run them on inputs of size two, how much longer do things take?

rather than look at implementations, we need to think mathematically.

attempt 2: better than brute force let’s start by making an agreement:

we will focus on worst-case performance. we want to find a bound on the largest possible

running time an algorithm could have, over all inputs of size n.

this has a nice property: not sensitive to inputs we choose.

next, let’s define “good” as “better than a brute-force search through all possible solutions.”

brute-force search most problems have a natural “solution

space” sorting path finding in graphs most optimization problems

with discrete (finite) problems, we can often just look at every possible solution to see if it’s what we want.

this is usually not tractable. solution spaces tend to grow quickly in sorting, brute force requires looking at n!

possible arrangements.

what’s wrong with this definition? “good” as “better than brute force” is

certainly better than our first attempt, but it’s not perfect.

main problem it’s too vague

we solve vagueness by making our definition more mathematical.

attempt 3: polynomial time in brute-force search, when we increase the

problem size by one, we increase the number of possibilities multiplicatively. in sorting, we go from n! to (n+1)!

a good algorithm should scale better. if we double the input size, we’d like the algorithm

to slow down by only a constant factor. we get this behavior by agreeing that good

algorithms are efficient, and an algorithm is efficient if its running time is

bounded above by some fixed polynomial.

terminology if an algorithm’s running time is bounded above

by a fixed polynomial, we may say: the algorithm has a polynomial running time. the algorithm runs in polynomial time. the algorithm is a polynomial-time algorithm.

if a problem has an algorithmic solution that runs in polynomial time, we might say: the problem is in the complexity class P. the problem is in P.

For instance, you probably know that sorting is in P. bubble sort’s running time is bounded above by n^10.

our goal: revised based on this discussion, we can be more

precise about the main goals of this course: we want to find polynomial-time algorithms for

important problems. we want to be able to prove that our algorithms

run in polynomial time. we also want to answer the questions

are there interesting problems (that are solvable) that are not in P?

for such problems, what do we do?

extras this is a theory course

I’ll provide opportunities to delve into more theoretical items as the course progresses.

optional for undergrads, less optional for grads. this is a practical course

you should be able to use this stuff every day. most of what we’re learning is at the core of the C+

+ standard library & the boost library. as the course progresses, we’re going to learn how to use

the standard library to do a lot for us. we’re also going to cover C++11 (the standard

formerly known as C++0x) algorithmic portions only – no variadic template craziness.

the schedule: main topics basics

mathematical foundations basic analysis

easy problems graph algorithms (discrete) optimization – dynamic programming &

greedy hard problems

intractability randomness and approximation

concurrency

other stuff we’ll cover a few topics that are not directly

related to algorithms, but are useful to know. unit testing – to make grading easier. profiling – to figure out what parts of your program

are taking so long. static & dynamic analysis tools – to find bugs and

general badness in your code.

extra #1: std::vector assignment #0

full instructions online download assignment0.cpp from the course webpage. make sure you can compile it.

the program illustrates the use of std::vector, which you can think of as a dynamically-resizable array.

you should get in the habit of using std::vector whenever you need a contiguous block of storage.

basic usage is similar to arrays, and is illustrated in the program.

from now on in this class, no more raw arrays.

next time: math review hopefully I’ve convinced you that we need to

do some math. the next few lectures will review the most

important ideas and techniques that we have to apply to prove that algorithms run in polynomial time.

also, assignment 0 is due. Assuming we can get g++ to work in the ECC.

Questions?