0. O-Notation

Die Korrektheit eines Algorithmus (für alle inputs) kann mithilfe eines Beweises festgestellt werden.

Die Laufzeit wird (unabhängig von dem jeweiligen Computer auf dem das Programm ausgeführt wird) mit dem Modell des asymptotischen Wachstums ermittelt.

Wächst $f(n)$ asymptotisch genauso schnell oder langsamer (${\lim }_{n\to \infty }\frac{f(n)}{g(n)}=CC\ne 0$) als $g(n)$ kann man schreiben, dass $f\in O(g)$.

Das relative Wachstum von $f$ ist also in Relation zu $g$ beschränkt. Dies sagt aber nichts über eine untere Grenze aus, also gilt z.B. $n\in O(n^{1000})$.

O-Notation

To write that $f\in O(g)$, we can also write that $f\le O(g)$. This is the case if there exists some constant $C$ that $\forall n$ ensures that $f(n)\le C\cdot g(n)$.

Big $O$ shortcuts

• If $\frac{f(n)}{g(n)}$ tends to $0$, then $f\le O(g)$ and $g\nleqq O(f)$
• If $\frac{f(n)}{g(n)}$ tends to $C\in {\mathbb{R}}^{+}$, then $f\le O(g)$ and $g\le O(f)$
• If $\frac{f(n)}{g(n)}$ tends to $\infty +$, then $f\nleqq O(g)$ and $g\le O(f)$

If $f\le O(h)$ and $g\le O(h)$

• $\forall C>0$, $c\cdot f\le O(h)$
• $f+g\le O(h)$

L’Hôpital’s Rule

L'Hôpital's Rule

Assume that functions $f:{\mathbb{R}}^{+}\to {\mathbb{R}}^{+}$ and $g:{\mathbb{R}}^{+}\to {\mathbb{R}}^{+}$ are differentiable, ${\lim }_{x\to \infty }f(x)={\lim }_{x\to \infty }g(x)=\infty$ and $\forall x\in {\mathbb{R}}^{+},g^{\prime }(x)\ne 0$. If ${\lim }_{x\to \infty }\frac{f^{\prime }(x)}{g^{\prime }(x)}=C\in {\mathbb{R}}_0^{+}$ or ${\lim }_{x\to \infty }\frac{f^{\prime }(x)}{g^{\prime }(x)}=\infty$, then: ${\lim }_{x\to \infty }\frac{f(x)}{g(x)}={\lim }_{x\to \infty }\frac{f^{\prime }(x)}{g^{\prime }(x)}$

Keep these limitations for using l’Hôpital’s rule in mind!

Asymptotic Notation

$\Omega$-Notation for lower bounds.

For $f:\mathbf{N}\to {\mathbb{R}}^{+}$,

$$\begin{array}{rl}\Omega (f)& :=\{g:\mathbb{N}\to {\mathbb{R}}^{+}\mid f\le O(g)\}\\ & =\{g:\mathbb{N}\to {\mathbb{R}}^{+}\mid \exists C\ge 0\forall n\in \mathbb{N}g(n)\ge C\cdot f(n)\}.\end{array}$$

We can write $g\ge \Omega (f)$ instead of $g\in \Omega (f)$.

$\Theta$-Notation for upper and lower bounds.

For $f:\mathbf{N}\to {\mathbb{R}}^{+}$,

$$\begin{array}{rl}\Theta (f)& :=\{g:\mathbb{N}\to {\mathbb{R}}^{+}\mid g\le O(f)\text{and}f\le O(g)\}\\ & =\{g:\mathbb{N}\to {\mathbb{R}}^{+}\mid \exists C_1,C_2\ge 0\forall n\in \mathbb{N}{C}_1\cdot f(n)\le g(n)\le C_2\cdot f(n)\}.\end{array}$$

We can write $g=\Theta (f)$ instead of $g\in \Theta (f)$.

We thus have:

$g\ge \Omega (f)\iff f\le O(g)$ and $g=\Theta (f)\iff g\le O(f)\text{and}f\le O(g)$ Which can be used to restate the previous theorem like this:

Big $O$ shortcuts

• If $\frac{f(n)}{g(n)}$ tends to $0$, then $f\le O(g)$ but $f\ne \Theta (g)$
• If $\frac{f(n)}{g(n)}$ tends to $C\in {\mathbb{R}}^{+}$, then $f=\Theta (g)$
• If $\frac{f(n)}{g(n)}$ tends to $\infty +$, then $f\ge \Omega (g)$ but $f\ne \Theta (g)$

The Master Theorem

To find the runtime of a recursively defined count of operations, you can use the master theorem.

Comparisons of Growth Rates for common functions

See this big inequality: $O(1)\le O(\log (n))\le O(n)\le O(n\log (n))\le O(n^k)\le O(k^n)\le O(n!)\le O(n^n)$

We also know that:

• $\log (n)\le O(\sqrt{n})$
• $\log (n!)\le O(n\log n)$

Deriving upper and lower bounds for formulas

We can quickly derive upper and lower bounds for some sums or functions using a trick:

• The upper limit can be expressed as the highest terms, times the amount of terms: ${\sum }_{i=1}^n{i}^3=1^3+2^3+3^3+...+n^3\le n\cdot {\sum }_{i=1}^n{n}^3=n^4$
• A similar trick works for the lower limit. Take a limited number of terms, which is then automatically lower than the sum. $\frac{n^4}{2^4}=\frac{n}{2}\cdot (\frac{n}{2}{)}^3={\sum }_{i=\frac{n}{2}}^n(\frac{n}{2}{)}^3\le {\sum }_{i=1}^n{i}^3=1^3+...+(\frac{n}{2}{)}^3+...+n^3$