How long with it run?

III-1-2-Asymptotics

How long with it run?

for i in 1 to 8
  for j in 1 to 4
     // do something
i on the x-axis
j on the y-axis

Steps in a program can be estimated as areas.

\(8\times 4=32\) steps total

Rule 1:

for i in 1 to m
  for j in 1 to n
     // do something
i on the x-axis, width n
j height n

Steps in a program can be estimated as areas.

\(m\times n=mn\) steps total

\sum_{i=1}^n \sum_{j=1}^m 1=m\times n

1=one unit of work, the "something"

How long with it run?

for i in 1 to 5
  for j in 1 to i
     // do something
i on the x-axis, width n
j height n

Steps in a program can be estimated as areas.

\(5+4+3+2+1=15=\frac{6\cdot 5}{2}\approx \frac{5\cdot 5}{2}\) steps total

How long with it run?

for i in 1 to n
  for j in 1 to i
     // do something
i on the x-axis, width n
j height n

Steps in a program can be estimated as areas.

\(n+\cdots +5+4+3+2+1=15=\frac{(n+1)\cdot n}{2}\approx \frac{n\cdot n}{2}\) steps total

Rule 2

for i in 1 to n
  for j in 1 to i
     // do something
i on the x-axis, width n
j height n

Steps in a program can be estimated as areas.

\sum_{i=1}^n \sum_{i=1}^n 1 =\frac{n^2}{2}+\text{ smaller digit terms }

Why does area work?

for i in 1 to n
  for j in i to n
     // do something
i on the x-axis, width n
j height n
\sum_{i=1}^n \sum_{i=1}^n 1 =\frac{n^2}{2}+\text{ smaller digit terms }

Goal of Asymptotic: simplify behavior for long term.

\frac{(n+1)n}{2}=\frac{n^2}{2}+\frac{n}{2}
\frac{n^2}{2} \gg \frac{n}{2}

If \(n\) has 10 digits then \(n^2\) has 20 digits.

20 digit number + 10 digit number is a 20 or 21 digit number.

The 10 digit number is huge but hardly makes a dent on the 20 digits!

Here \(\gg\) reads "much much bigger"

So area might miss some actual values, like \(n/2\) but those are minute impacts in the long run.

Area formula

"If \(n\) has 10 digits then \(n^2\) has 20 digits..... blah blah blah..."

 

Nice but vague and open to criticism.  

Maybe you do care about 10 digit changes.

 

Better to use a limit explanation!

Asymptotic: the limit method.

\lim_{n\to\infty}\frac{f(n)}{\alpha n^k}=1

If \(f(n)\) is the real count and we want it to be a simpler \(\alpha n^k\) then up to some margin for error we accept \(f(n)\) as equal to \(\alpha n^k\), with possibly finitely many exceptions.

A LIMIT!

So we just need to find \(\alpha\).  And since we know the most we do is independent choices for each loop, if we count the nested loops as \(k\) then \(\alpha n^k\) is the target.  

Asymptotic: the limit method.

\frac{1}{n^2}\frac{(n+1)n}{2}=\frac{n^2}{2n^2}+\frac{n}{2n^2}=\frac{1}{2}+\frac{1}{2n}

1. Divide by highest power, here \(n^2\).

\begin{aligned} \lim_{n\to \infty}\frac{1}{n^2}\frac{(n+1)n}{2} & =\lim_{n\to\infty}\left(\frac{1}{2}+\frac{1}{2n}\right)\\ & =\lim_{n\to\infty}\frac{1}{2}+\lim_{n\to\infty}\frac{1}{2n}\\ & =\frac{1}{2}+0=\frac{1}{2}=\alpha\\ \end{aligned}

2. Take the limit as \(n\to \infty\) "goes towards infinity".

3. Highest power times this limit is your asymptote:  \(\frac{1}{2}n^2\)

Rule A+B (Distributive)

for s in 1 to m
 // do this 
 
for i in 1 to n
  for j in 1 to i
     // do that
i width n
j height n

Steps in a program can be estimated as areas.

Time(A+B) = Time(A)+Time(B)
s length m

Example

for s in 1 to 100
	for t in 1 to s
		 // do this 
 
for i in 1 to 40
  for j in 1 to 50
  	  for k in 1 to 20
	     // do that
s width 100
t height 100

Steps in a program can be estimated as areas & volumes.

\begin{aligned} Time(A+B) & = Time(A)+Time(B)\\ & = \frac{100^2}{2}+ 40\cdot 50\cdot 20\\ & = 5000+40000=45000 \end{aligned}
i 40
k 20
j 50

How long with it run?

for i in 1 to n
  for j in 1 to i
     for k in 1 to j
     // do something

Steps in a program can be estimated as volumes.

What on earth is the volume of that!?

Some of you might see you can copy this shape 6 times to make a cube.

By own work - Own work, Public Domain, https://commons.wikimedia.org/w/index.php?curid=11978055
j
i
k

If you're that geometric you could answer \(\approx \frac{n^3}{6}\).

How long with it run?

for i in 1 to 5
  for j in 1 to i
     for k in 1 to j
     // do something

Steps in a program can be estimated as volumes.

What on earth is the volume of that!?

You don't need to know the exact 

j
i
k

Since \(1\leq k\leq j\leq i\leq n\). 

  • So you have \(n^3\) ways to choose \(i,j,k\) independently.
  • You may need to sort them in order, 3 ways to move one, 2 ways to move a second, so divide by \(3\times 2\times 1=6\).

Rule 3

for i in 1 to 5
  for j in 1 to i
     for k in 1 to j
     // do something

Steps in a program can be estimated as volumes.

j
i
k
\sum_{i=1}^n\sum_{j=1}^i \sum_{k=1}^j 1=\frac{n^3}{3\cdot 2\cdot 1}+\text{ smaller digit terms }

Rule n

for i_1 in 1 to n
   for i_2 in 1 to i_1
      ...
        for i_k in 1 to i_(k-1)
		      // do something
\sum_{i_1=1}^n\cdots \sum_{i_k=1}^{i_{k-1}} 1=\frac{n^k}{k!}+\text{ smaller digit terms }

\(k!=k\cdot (k-1)\cdots 2\cdot 1\)

Asymptotics

By James Wilson

Asymptotics

How long will your program run? What will stand out in your data? Asymptotics can give us answers.

  • 19

More from James Wilson