mastering algorithms, space and time complexity the p vs np problem understand that algorithms can...

Mastering Algorithms, space and time complexity

• The P vs NP Problem• Understand that algorithms can be compared by expressing their • complexity as a function relative to the size of the problem.• Understand that some algorithms are more efficient time-wise than other algorithms.• Understand that some algorithms are more space-efficient than other algorithms.• Big-O notation Linear time, polynomial time, exponential time.• Python (try it yourself tasks)• Order of complexity

Ever heard of the P vs NP Problem?Ever heard of the P vs NP Problem?

Before we do anything else, let me introduce you to this fabulous problem -it remains (at the time of writing) one of the major unsolved problems in Computer Science.You could be the one to solve it!

Before we do anything else, let me introduce you to this fabulous problem -it remains (at the time of writing) one of the major unsolved problems in Computer Science.You could be the one to solve it!

But first …what is it?What does it have to do with computing and algorithms?All these questions will be answered!

But first …what is it?What does it have to do with computing and algorithms?All these questions will be answered!

Why does the question matter?Why does the question matter?The P versus NP problem is a major unsolved problem in computer science. The P versus NP problem is a major unsolved problem in computer science.

Informally, it asks whether every problem whose solution can be quickly verified by a computer can also be quickly solved by a computer.

Informally, it asks whether every problem whose solution can be quickly verified by a computer can also be quickly solved by a computer.

It was introduced in 1971 by Stephen Cook in his seminal paper "The complexity of theorem proving procedures" and is considered by many to be the most important open problem in the field.

It was introduced in 1971 by Stephen Cook in his seminal paper "The complexity of theorem proving procedures" and is considered by many to be the most important open problem in the field.

It is one of the seven Millennium Prize Problems selected by the Clay Mathematics Institute to carry a US$ 1,000,000 prize for the first correct solution.

It is one of the seven Millennium Prize Problems selected by the Clay Mathematics Institute to carry a US$ 1,000,000 prize for the first correct solution.

And it gets better ….And it gets better ….

Aside from being an important problem in computational theory, a proof either way would have profound implications for mathematics, Philosophy, cryptography, algorithm research, artificial intelligence, game theory, multimedia processing and many other fields.

Aside from being an important problem in computational theory, a proof either way would have profound implications for mathematics, Philosophy, cryptography, algorithm research, artificial intelligence, game theory, multimedia processing and many other fields.

Really, if a solution is discovered, it could change the world as we know it!Really, if a solution is discovered, it could change the world as we know it!

There is a subset of computer science called complexity theory and one of the main questions that is asked in P vs NP is: “How powerful can computers really be?”

There is a subset of computer science called complexity theory and one of the main questions that is asked in P vs NP is: “How powerful can computers really be?”

What do you think? Imagine an algorithm that could literally solve any problem. Hmmm…..What do you think? Imagine an algorithm that could literally solve any problem. Hmmm…..

Decision problemsDecision problemsLife is full of all kinds of problems. Relationship problems, money problems…and I could go on. But in Maths and Computing there is a type of problem called DECISION PROBLEMS

Life is full of all kinds of problems. Relationship problems, money problems…and I could go on. But in Maths and Computing there is a type of problem called DECISION PROBLEMS

That is does x have a given property and can we signal this fact by the output of a “yes” or “no”

(x = given set of data)

That is does x have a given property and can we signal this fact by the output of a “yes” or “no”

(x = given set of data)

INPUT

Yes No

Algorithm

??

Here’s an example:Here’s an example:

EVEN-SUM Problem

"is the sum of 2, 3, 5, and 9 even?"

Here the answer would be … nope!

That’s because 2 + 3 + 5 + 9 = 19, which is an odd number.

EVEN-SUM Problem

"is the sum of 2, 3, 5, and 9 even?"

Here the answer would be … nope!

That’s because 2 + 3 + 5 + 9 = 19, which is an odd number.

??

??

INPUT

Yes No

Algorithm

Enter 2,3,5,9Enter 2,3,5,9

Is the sum of the inputs EVEN?

Is the sum of the inputs EVEN?

NoNoxx

Given a problem like the EVEN-SUM …Given a problem like the EVEN-SUM …

The goal is to create a method by which to solve any instance of the problem. This method is called an algorithm, and an algorithm must work correctly on every possible input to be considered correct. A decision problem may have multiple algorithms that solve it. For

example, two different algorithms can solve EVEN-SUM:

The goal is to create a method by which to solve any instance of the problem. This method is called an algorithm, and an algorithm must work correctly on every possible input to be considered correct. A decision problem may have multiple algorithms that solve it. For

example, two different algorithms can solve EVEN-SUM:

1. add up all the numbers and output Yes if the sum is even or No if the sum is not even

1. add up all the numbers and output Yes if the sum is even or No if the sum is not even

2. output Yes if an even number of the input numbers are odd, or No otherwise.

Which method is better? Time efficient?Which method is better? Time efficient?Both of them solve the problem, but one of them is more time efficient,

and cleverer – doing less work along the way!Both of them solve the problem, but one of them is more time efficient,

and cleverer – doing less work along the way!

1. add up all the numbers and output Yes if the sum is even or No if the sum is not even1. add up all the numbers and output Yes if the sum is even or No if the sum is not even

2. output Yes if an even number of the input numbers are odd, or No otherwise.

Instead of adding up all the numbers, the second algorithm makes use of the fact that the sum of two odd numbers is even! Genius!

Instead of adding up all the numbers, the second algorithm makes use of the fact that the sum of two odd numbers is even! Genius!

The second algorithm is more efficient than the first in the sense that as the size of the input increases, the number of steps required to solve the problem increases more slowly.


Let’s look at those key words again:Let’s look at those key words again:



??

??

??

??

Complexity ClassesComplexity Classes

Computer scientists place decision problems in complexity classes by how efficient their most efficient known algorithm is.

The complexity class P, or P for short, is the class of decision problems that we know how to solve with reasonable efficiency.

Computer scientists place decision problems in complexity classes by how efficient their most efficient known algorithm is.

The complexity class P, or P for short, is the class of decision problems that we know how to solve with reasonable efficiency.

Complexity ClassesComplexity Classes

Similarly, NP is a complexity class that contains all problems that can be solved reasonably efficiently given an extra bit of information called a certificate or hint

Similarly, NP is a complexity class that contains all problems that can be solved reasonably efficiently given an extra bit of information called a certificate or hint

As an example, take the classic Travelling Salesman Problem, or TSP. An instance of TSP is something like "does there exist a route that passes through Chicago, Manhattan, Boston, and Miami exactly once each that is no more than 2000 miles long?"

As an example, take the classic Travelling Salesman Problem, or TSP. An instance of TSP is something like "does there exist a route that passes through Chicago, Manhattan, Boston, and Miami exactly once each that is no more than 2000 miles long?"

DefinitionsDefinitions

= the Complexity Class (or P for short) – it is the class ofDecision problems that we know how to solve with reasonableefficiency

= the Complexity Class (or P for short) – it is the class ofDecision problems that we know how to solve with reasonableefficiency

= Complexity class that contains all problems that can be solvedReasonable efficiently given an extra bit of information (this Extra bit is called a certificate or hint)

= Complexity class that contains all problems that can be solvedReasonable efficiently given an extra bit of information (this Extra bit is called a certificate or hint)

The travelling salesman problem (TSP)The travelling salesman problem (TSP)

Let’s look more closely at this famous problem. And to make it more interesting, let’s call it the travelling pig problem …

Let’s look more closely at this famous problem. And to make it more interesting, let’s call it the travelling pig problem …

An instance of this problem would be something like: “Does there exist A route that passes through Cities Q, R, S, and T exactly once each that is no more than 2000 miles long?”

An instance of this problem would be something like: “Does there exist A route that passes through Cities Q, R, S, and T exactly once each that is no more than 2000 miles long?”

We are asking the question: “Does there exist a route that passes through Cities Q, R, S, and T exactly once each that

Is no more than 2000 miles long?”



1) Try all possible orderings of the four cities and compute the length of each route

The travelling pig problem….The travelling pig problem….





What if you were given a *hint*?!What if you were given a *hint*?!

Suppose you were given a Hint such as : Try Going via Q R S T

WHEN the Hint is provided: A valid Algorithm only needs now to check the length of the suggested (hint) path And output YES if the path is short enough.


1) Try all possible orderings of the four cities and compute the length of each route1) Try all possible orderings of the four cities and compute the length of each route

Take the hint, find the solution!Take the hint, find the solution!

The hint given to you is : Try Going via Q R S T



DefinitionsDefinitions

P = the Complexity Class (or P for short) – it is the class of Decision

problems that we know how to solve with reasonable efficiency



NP =Complexity class that contains all problems that can be solved

reasonable efficiently given an extra bit of information (this extra bit is called a certificate or hint)



= algorithm which is reasonably efficient but has been given a hint

*Note that TSP is in NP (it can be solved efficiently if given a certificate)

Note: TSP is in NPNote: TSP is in NP





Is in NP

Is interestingly, also in NP (as algorithms for problems in P can simply ignore the certificate handed to them)





= There are no known (reasonably efficient) algorithms that do not need a HINT (or certificate)

WHEN the Hint is provided: A valid algorithm only needs now to check the length of the suggested (hint) path And output YES if the path is short enough.

You could try the most obvious algorithm – which is “try all possible paths” – but this becomes terribly terribly slow when C (no of cities) is even a 100! Believe it or not the number of possible paths is a 157 digit number and trying all of these paths would take literally years on even the fastest computer!

Consider further …Consider further …

If no known algorithms are any betterIf no known algorithms are any better

Can we really say that TSP is in P?Can we really say that TSP is in P?

Not really! (because for TSP to be in P it would have to have some known algorithm that can be executed with reasonable efficiency and no known algorithm has been discovered. Yet – logically, it appears that P (as shown on the previous slide) is in NP (because anything in P can simply ignore the certificate). Logically, then TSP is in P – but really ….in reality (according to the evidence we have) it isn’t

Not really! (because for TSP to be in P it would have to have some known algorithm that can be executed with reasonable efficiency and no known algorithm has been discovered. Yet – logically, it appears that P (as shown on the previous slide) is in NP (because anything in P can simply ignore the certificate). Logically, then TSP is in P – but really ….in reality (according to the evidence we have) it isn’t

So do we give up because we don’t have thee evidence (or rather the power to compute the evidence)So do we give up because we don’t have thee evidence (or rather the power to compute the evidence)

How can we prove that TSP is NOT in P?How can we prove that TSP is NOT in P?

The question of P vs NP considers:The question of P vs NP considers:

1. Is the Class P the SAME AS NP

2. Are we able to show that P is NOT the same as NP?

1. Is the Class P the SAME AS NP

2. Are we able to show that P is NOT the same as NP?

On the other hand P = NP is easy to ‘show’On the other hand P = NP is easy to ‘show’

• TSP is a special kind of NP Problem called an NP-Complete Problem

• All NP-Complete Problems are equivalent to each other

• TSP is a special kind of NP Problem called an NP-Complete Problem

• All NP-Complete Problems are equivalent to each other

How do we show P = NP?How do we show P = NP?

All we need is ONE efficient algorithm for ANY NP-Complete Problem

We could of course just assume that P is NOT the same as NP, but assumptions aren’t what we’re looking for here …!

All we need is ONE efficient algorithm for ANY NP-Complete Problem

We could of course just assume that P is NOT the same as NP, but assumptions aren’t what we’re looking for here …!

•A decision problem C is NP-complete if:1. C is in NP, and

2.Every problem in NP is reducible in polynomial time.

3.can be shown to be in NP by demonstrating that a candidate solution to can be verified in polynomial time.

4.Note that a problem satisfying condition 2 is said to be NP-hard whether or not it satisfies condition 1.

Formal definition of NP-CompletenessFormal definition of NP-Completeness

Polynomial time?Polynomial time?

Polynomial time is a synonym for "tractable", "feasible", "efficient", or "fast"Polynomial time is a synonym for "tractable", "feasible", "efficient", or "fast"

If a problem cannot be solved in polynomial time, it is …er….not going to be easy to solve!If a problem cannot be solved in polynomial time, it is …er….not going to be easy to solve!

Some examples of polynomial time algorithms:

•The quicksort sorting algorithm on n integers performs at most An^2 operations for some constant A. Thus it runs in time O(n^2) and is a polynomial time algorithm.•All the basic arithmetic operations (addition, subtraction, multiplication, division, and comparison) can be done in polynomial time.•Maximum matchings in graphs can be found in polynomial time.

Some examples of polynomial time algorithms:

•The quicksort sorting algorithm on n integers performs at most An^2 operations for some constant A. Thus it runs in time O(n^2) and is a polynomial time algorithm.•All the basic arithmetic operations (addition, subtraction, multiplication, division, and comparison) can be done in polynomial time.•Maximum matchings in graphs can be found in polynomial time.

Problems that cannot be solved in Polynomial time?

Problems that cannot be solved in Polynomial time?

NP stands for Non-deterministic Polynomial time.: This means that the problem can be solved in Polynomial time using a Non-deterministic Turing machineNP stands for Non-deterministic Polynomial time.: This means that the problem can be solved in Polynomial time using a Non-deterministic Turing machine

Although any given solution to an NP-complete problem can be verified quickly (in polynomial time), there is no known efficient way to locate a solution in the first place; indeed, the most notable characteristic of NP-complete problems is that no fast solution to them is known. .

Although any given solution to an NP-complete problem can be verified quickly (in polynomial time), there is no known efficient way to locate a solution in the first place; indeed, the most notable characteristic of NP-complete problems is that no fast solution to them is known. .

That is, the time required to solve the problem using any currently known algorithm increases very quickly as the size of the problem grows. As a consequence, determining whether or not it is possible to solve these problems quickly, called the P versus NP problem, is one of the principal unsolved problems in computer science today!

• Boolean satisfiability problem (Sat.)

• N-puzzle

• Knapsack problem

• Hamiltonian path problem

• Travelling salesman problem

• Subgraph isomorphism problem

• Subset sum problem

• Clique problem

• Vertex cover problem

• Independent set problem

• Dominating set problem

• Graph coloring problem

Examples of other Problems that are NP-Complete when expressed as Decision problems … if you’re interested click the links to read more!

Examples of other Problems that are NP-Complete when expressed as Decision problems … if you’re interested click the links to read more!

http://en.wikipedia.org/wiki/Boolean_satisfiability_problem

http://en.wikipedia.org/wiki/N-puzzle

http://en.wikipedia.org/wiki/Knapsack_problem

http://en.wikipedia.org/wiki/Hamiltonian_path_problem

http://en.wikipedia.org/wiki/Travelling_salesman_problem

http://en.wikipedia.org/wiki/Subgraph_isomorphism_problem

http://en.wikipedia.org/wiki/Subset_sum_problem

http://en.wikipedia.org/wiki/Clique_problem

http://en.wikipedia.org/wiki/Vertex_cover_problem

http://en.wikipedia.org/wiki/Independent_set_problem

Big O Notation (Mathematics)Big O Notation (Mathematics)

In mathematics, big O notation describes the limiting behaviour of a function when the argument tends towards a particular value or infinity, usually in terms of simpler functions

In mathematics, big O notation describes the limiting behaviour of a function when the argument tends towards a particular value or infinity, usually in terms of simpler functions

It is a member of a larger family of notations that is called Landau notation, Bachmann–Landau notation (after Edmund Landau and Paul Bachmann),or asymptotic notation.

It is a member of a larger family of notations that is called Landau notation, Bachmann–Landau notation (after Edmund Landau and Paul Bachmann),or asymptotic notation.

Big O Notation (Computer Science)Big O Notation (Computer Science)

In computer science, big O notation is used to classify algorithms by how they respond (e.g., in their processing time or working space requirements) to changes in input size.

In computer science, big O notation is used to classify algorithms by how they respond (e.g., in their processing time or working space requirements) to changes in input size.

A famous example is the problem of estimating the remainder term in the prime number theorem.

A famous example is the problem of estimating the remainder term in the prime number theorem.

This may sound very high flung and complex, but the basic concepts associated with the Big O Notation are not hard at all.

This may sound very high flung and complex, but the basic concepts associated with the Big O Notation are not hard at all.

To put it simply, Big-O Notation is how programmers talk about algorithms. *think, in this case, of an algorithm as a function that occurs in your program.

To put it simply, Big-O Notation is how programmers talk about algorithms. *think, in this case, of an algorithm as a function that occurs in your program.

Big O Notation continued: Big O Notation continued:

A function’s Big-O Notation is basically determined by how it responds to different inputs

A function’s Big-O Notation is basically determined by how it responds to different inputs

You might, for instance, ask the question: How much slower would the program be if we gave it a list of 1000 things to work on instead of a list of just 1 thing?

You might, for instance, ask the question: How much slower would the program be if we gave it a list of 1000 things to work on instead of a list of just 1 thing?

Consider the block of code on the top right hand corner:Consider the block of code on the top right hand corner:

If we call this function in the following manner:

item_in_list(2, [1,2,3]),

The response would be rather quick! You are basically looping over each thing in the list and if you find the first argument to your function, then return True. If we get to the end and it has not been found, return False.

If we call this function in the following manner:


The response would be rather quick! You are basically looping over each thing in the list and if you find the first argument to your function, then return True. If we get to the end and it has not been found, return False.

Function xFunction x


If 2 is found, then return True.

If end of list is reached and 2 is not found, return False


If 2 is found, then return True.

If end of list is reached and 2 is not found, return False

Function xFunction x

The complexity of this function is O(n)The complexity of this function is O(n)

This is read “Order of n” as the O function is also known as

the Order function (to do with approximation which deals with ‘orders of magnitude’

This is read “Order of n” as the O function is also known as

the Order function (to do with approximation which deals with ‘orders of magnitude’

Orders of MagnitudeOrders of Magnitude

A jar whose number of sweets is probably within an order of

magnitude of 100

A jar whose number of sweets is probably within an order of

magnitude of 100

O(n): Consider furtherO(n): Consider further

if we were to graph the time it takes to run this function with different sized inputs (e.g. an array of 1 item, 2 items, 3 items, etc.), we'd see that it approximately corresponds to the number of items in the array

if we were to graph the time it takes to run this function with different sized inputs (e.g. an array of 1 item, 2 items, 3 items, etc.), we'd see that it approximately corresponds to the number of items in the array

This is called a linear graph. This means that the line is basically straight if you were to graph it.

This is called a linear graph. This means that the line is basically straight if you were to graph it.

O(n): Consider furtherO(n): Consider further

Note: In the code above, if the item we were looking for was always the first item in the list – the function would be super efficient!

Note: In the code above, if the item we were looking for was always the first item in the list – the function would be super efficient!

Big O however is all about considering the WORST CASE SCENARIO! (the worst-case performance of doing something)

Big O however is all about considering the WORST CASE SCENARIO! (the worst-case performance of doing something)

In this example, the worst case is that …. the item we are looking for isn’t in the list at all!(The math term for this is "upper bound").

In this example, the worst case is that …. the item we are looking for isn’t in the list at all!(The math term for this is "upper bound").

Plotting an O(n) graphPlotting an O(n) graph

Runtime characteristics of an O(n) function.

Runtime characteristics of an O(n) function.

Runtime characteristics of an O(1) function.

Runtime characteristics of an O(1) function.

Recap on definitionsRecap on definitions

Big O notation is the language we use for articulating how long an algorithm takes to run. It's how we compare the efficiency of different approaches to a problem

Big O notation is the language we use for articulating how long an algorithm takes to run. It's how we compare the efficiency of different approaches to a problem

With big O notation we express the runtime in terms of—brace yourself for a techy sounding sentence—how quickly it grows relative to the input, as the input gets arbitrarily large.

With big O notation we express the runtime in terms of—brace yourself for a techy sounding sentence—how quickly it grows relative to the input, as the input gets arbitrarily large.

Let’s look at three things (from the above sentence) more closely:1. How quickly the runtime grows2. Relative to the input3. As the input gets arbitrarily large.

Let’s look at three things (from the above sentence) more closely:1. How quickly the runtime grows2. Relative to the input3. As the input gets arbitrarily large.

??

?? ??

??

??

?? ?? ??

How quickly the runtime “grows”How quickly the runtime “grows”

Often the case is that external factors affect the time it takes for a function to run. This could include things like-the speed of the processor-other programs the computer is running etc.

Often the case is that external factors affect the time it takes for a function to run. This could include things like-the speed of the processor-other programs the computer is running etc.

For the above reason, it's hard to make strong statements about the exact runtime of an algorithm. For the above reason, it's hard to make strong statements about the exact runtime of an algorithm.

Instead we use big O notation to express how quickly its runtime growsInstead we use big O notation to express how quickly its runtime grows

Relative to the inputRelative to the input

Note that it is not an exact number that we are looking for. What we need is something to phrase our runtime growth in terms of.

Note that it is not an exact number that we are looking for. What we need is something to phrase our runtime growth in terms of.

Or "on the order of the square of the size of the inputOr "on the order of the square of the size of the input

It’s quite useful to use the size of the input. We can say things like the runtime grows "on the order of the size of the input" It’s quite useful to use the size of the input. We can say things like the runtime grows "on the order of the size of the input"

As the input gets arbitrarily large …As the input gets arbitrarily large …

You’ve got to remember that your algorithm may seem expensive or appear to have too many steps when n is small but this is made up for eventually by other steps as n gets huge.

You’ve got to remember that your algorithm may seem expensive or appear to have too many steps when n is small but this is made up for eventually by other steps as n gets huge.

When working with Big-O analysis, we are most concerned with the stuff that grows fastest as the input grows, because everything else is quickly eclipsed as n gets very large.

When working with Big-O analysis, we are most concerned with the stuff that grows fastest as the input grows, because everything else is quickly eclipsed as n gets very large.

If you know what an asymptote is, you might see why "big O analysis" is sometimes called "asymptotic analysis."

If you know what an asymptote is, you might see why "big O analysis" is sometimes called "asymptotic analysis."

Big-O notation seems like an abstract concept, but it would help to look at some coded examples

Big-O notation seems like an abstract concept, but it would help to look at some coded examples

As we work through them, think about their relative time complexities. Assign each of the below one of the 5 categories

Excellent Good Fair Bad Horrible!

Example #1 (Try it yourself in Python)Example #1 (Try it yourself in Python)

Note: This function runs in O(1) time (or "constant time") relative to its input. The input list could be 1 item or 1,000 items, but this function would still just require one "step.“

It will always, always only return the first item on the list.

Note: This function runs in O(1) time (or "constant time") relative to its input. The input list could be 1 item or 1,000 items, but this function would still just require one "step.“

It will always, always only return the first item on the list.

Excellent


This function runs in O(n) time (or "linear time"), where n is the number of items in the list.

If the list has 10 items, it prints 10 items. If the list has 1 million items, well then 1 million items would be printed!

This function runs in O(n) time (or "linear time"), where n is the number of items in the list.

If the list has 10 items, it prints 10 items. If the list has 1 million items, well then 1 million items would be printed!

??

??


There are two nested loops here. If our list has n items, our outer loop runs n times and our inner loop runs n times for each iteration of the outer loop, giving us n^2 total prints. Thus this function runs in O(n^2) time (or "quadratic time"). The GROWTH of the solution is evident: If the list has 10 items, we have to print 100 times. If it has 1,000 items, we have to print 1,000,000 times.

There are two nested loops here. If our list has n items, our outer loop runs n times and our inner loop runs n times for each iteration of the outer loop, giving us n^2 total prints. Thus this function runs in O(n^2) time (or "quadratic time"). The GROWTH of the solution is evident: If the list has 10 items, we have to print 100 times. If it has 1,000 items, we have to print 1,000,000 times.??

Horrible!

Note: n could be the ACTUAL INPUT or n could be the ‘SIZE OF THE INPUT’Note: n could be the ACTUAL INPUT or n could be the ‘SIZE OF THE INPUT’

Note: sometimes n is an actual number that's an input into our function, and other times n is thenumber of items in an input array (or an input map, or an input object, etc.).

Note: sometimes n is an actual number that's an input into our function, and other times n is thenumber of items in an input array (or an input map, or an input object, etc.).

Further note that you can GET RID OF THE CONSTANTSFurther note that you can GET RID OF THE CONSTANTS

This is O(2n), which we just call O(n).

This is O(2n), which we just call O(n).

Further note that you can GET RID OF THE CONSTANTSFurther note that you can GET RID OF THE CONSTANTS

This is O(1+n/2+100) which we just call O(n)This is O(1+n/2+100) which we just call O(n)

How come getting rid of the constants doesn’t matter?How come getting rid of the constants doesn’t matter?

This is O(1+n/2+100) which we just call O(n)This is O(1+n/2+100) which we just call O(n)

Remember that for the Big O Notation we are concerned with what happens when n gets arbiritarily large.

Remember that for the Big O Notation we are concerned with what happens when n gets arbiritarily large.

In the example above we are doing things like adding 100 and dividing by two- and as n gets REALLLY big – these things have a decreasingly significant effect.

In the example above we are doing things like adding 100 and dividing by two- and as n gets REALLLY big – these things have a decreasingly significant effect.

We can also drop the less significant terms. (Try this yourself in Python) We can also drop the less significant terms. (Try this yourself in Python)

Here our runtime is O(n+n²)which we just call O(n²). Even if it was O(n²/2+100n), it would still be O(n²)

Here our runtime is O(n+n²)which we just call O(n²). Even if it was O(n²/2+100n), it would still be O(n²)

Linear, Quadratic and Exponential time:Linear, Quadratic and Exponential time:

Linear growthLinear growth

Quadratic growthQuadratic growth

ExponentialExponential

Linear growth occurs when there is a constant rate of change. The equation for a linear relationship is y = bx, and its graph is a straight line. If you work for an hourly wage, and don’t spend your earnings, your savings will grow linearly

Linear growth occurs when there is a constant rate of change. The equation for a linear relationship is y = bx, and its graph is a straight line. If you work for an hourly wage, and don’t spend your earnings, your savings will grow linearly

When the rate of change increases with time, numbers can grow more quickly. Quadratic and cubic growth can be represented by y = x2 and y = x3, respectively. The general form of this type of relationship can be written y = xb, and is called polynomial growth. The distance travelled by a falling object can be calculated with a quadratic equation.

When the rate of change increases with time, numbers can grow more quickly. Quadratic and cubic growth can be represented by y = x2 and y = x3, respectively. The general form of this type of relationship can be written y = xb, and is called polynomial growth. The distance travelled by a falling object can be calculated with a quadratic equation.LogarithmicLogarithmic

Linear, Quadratic and Exponential time:Linear, Quadratic and Exponential time:

ExponentialExponentialExponential (or geometric) growth is faster still. Here, the rate of growth is proportional to the value of y at any time. Exponential relationships can be expressed as y = bx. Bacterial populations with unlimited food, nuclear chain reactions, and computer processing power are all said to grow exponentially.

Exponential (or geometric) growth is faster still. Here, the rate of growth is proportional to the value of y at any time. Exponential relationships can be expressed as y = bx. Bacterial populations with unlimited food, nuclear chain reactions, and computer processing power are all said to grow exponentially.

Linear, Quadratic and Exponential time and LOGARITHMIC GrowthLinear, Quadratic and Exponential time and LOGARITHMIC Growth

LogarithmicLogarithmicLogarithmic growth is the inverse of exponential growth. Logarithmic phenomena grow very slowly, and have an equation of the form y = logbx. Sound volume and frequency are both perceived logarithmically, allowing humans to detect a huge range of sound levels.

Logarithmic growth is the inverse of exponential growth. Logarithmic phenomena grow very slowly, and have an equation of the form y = logbx. Sound volume and frequency are both perceived logarithmically, allowing humans to detect a huge range of sound levels.

Linear, Quadratic or Exponential time?Linear, Quadratic or Exponential time?

Order of ComplexityOrder of Complexity

You could think of order of complexity as referring to going from the simplest types to the more complex:

O(1), O(log n), O(n), O(n log n), O(n^2)

You could think of order of complexity as referring to going from the simplest types to the more complex:

O(1), O(log n), O(n), O(n log n), O(n^2)

O(log n) — An algorithm is said to be logarithmic if its running time increases logarithmically in proportion to the input size.O(n) — A linear algorithm’s running time increases in direct proportion to the input size.O(n log n) — A superlinear algorithm is midway between a linear algorithm and a polynomial algorithm.O(n^c) — A polynomial algorithm grows quickly based on the size of the input.O(c^n) — An exponential algorithm grows even faster than a polynomial algorithm.O(n!) — A factorial algorithm grows the fastest and becomes quickly unusable for even small values of n.


Order of ComplexityOrder of Complexity



??

??

??

??

??

??

??

Know thy complexity: http://bigocheatsheet.com/ Know thy complexity: http://bigocheatsheet.com/

New to these concepts? Brush up on your maths! New to these concepts? Brush up on your maths!

mastering algorithms, space and time complexity the p vs np problem understand that algorithms can...

Documents