Big-O

Appending to a Python list or JavaScript array almost always feels instantaneous, yet every so often the structure allocates a fresh buffer and copies every element into it, an `O(n)` operation. So why do textbooks still call append `O(1)`? Because amortized over a long sequence of appends, the rare expensive operation is paid for by many cheap ones. **Amortized Analysis** is the formal machinery that makes this kind of argument rigorous. This lesson teaches the three classical methods. The **aggregate method** sums the total cost of `n` operations and divides by `n` for a per-operation bound. The **accounting (banker's) method** charges each operation a fixed amount and stores credits on the data structure to pay for future expensive ones. The **potential (physicist's) method** defines a potential function `Phi` and computes amortized cost as actual cost plus the change in `Phi`. You will apply all three to dynamic array doubling, stacks with multipop, and binary counters, and see how amortized `O(1)` differs from worst-case `O(1)` and from average-case analysis. This lesson stands on the shoulders of **Big-O Notation (Upper Bound)** and **Time Complexity Analysis Techniques**. From those lessons you can already analyze a single operation; amortized analysis is what you reach for when the cost of *one* operation is misleading and the right object of study is a *sequence* of operations. Next you will move to **Recurrence Relations**, the other major analysis tool, used when an algorithm's cost is most naturally written recursively rather than as a running total.

How do you compare two solutions to the same problem without ever running them on real hardware? Big-O is the symbolic shorthand that the entire industry uses to do exactly that, and it is the single most referenced notation in textbooks, technical interviews, and engineering documentation. **Big-O Notation (Upper Bound)** turns the growth-rate ideas you have been thinking about into precise notation. You will see what `O(f(n))` formally means as an upper bound, walk through the seven complexity classes you will meet again and again (`O(1)`, `O(log n)`, `O(n)`, `O(n log n)`, `O(n^2)`, `O(2^n)`, `O(n!)`), and learn the rules for reading Big-O straight from code: single loops, nested loops, sequential blocks, conditional branches, and inputs with multiple variables like `O(n + m)` and `O(n * m)`. You will also meet best, worst, and average case analysis so you know which scenario a Big-O statement is describing. In **Asymptotic Analysis Fundamentals**, you saw why we count operations as a function of input size `n`, why constants and lower-order terms drop away, and how growth rates rank from constant to exponential. Big-O is the formal language for everything you reasoned about there: each growth-rate intuition becomes a concrete `O(...)` expression. Once you can classify code by sight, you will be ready for **Time Complexity Analysis Techniques**, where you will apply these rules to longer, more realistic snippets and develop a systematic analysis workflow.

When you can say *both* that an algorithm runs in `O(n log n)` and that it must do at least `Omega(n log n)` work, the ceiling and floor have collapsed onto the same growth rate. That coincidence is the most informative complexity statement you can make, and it is what **Big-Theta Notation** captures in a single symbol. This lesson defines `Theta(g(n))` as the intersection of Big-O and Big-Omega: there exist positive constants `c1`, `c2`, and `n0` such that `c1 * g(n) <= f(n) <= c2 * g(n)` for all `n >= n0`, the so-called sandwich property. You will practice classifying functions and algorithms by their tight bound, learn when Big-Theta is appropriate (best and worst case have the same growth rate) versus when only Big-O is honest (the cases differ), and see why merge sort is `Theta(n log n)` while linear search is *not* Theta of any single function. This lesson stitches together two ideas you have already met. From **Big-O Notation (Upper Bound)** you have the ceiling `f(n) <= c * g(n)`, and from **Big-Omega Notation (Lower Bound)** you have the floor `f(n) >= c * g(n)`. Big-Theta simply requires both bounds to hold for the same `g(n)`, so an algorithm only earns a Theta label when its upper and lower bounds genuinely match. With all three asymptotic notations in hand, you will be ready to switch focus to **Memory Models**, where the same growth-rate thinking gets applied to space rather than time.

Some problems have algorithms that finish in seconds on inputs of size a million. Others, like finding the shortest tour through 30 cities, can defeat the world's fastest computers no matter how cleverly you code. The boundary between these two worlds is studied formally in **complexity theory**, and the rough map of that boundary is given by **complexity classes**: `P`, `NP`, `NP-Complete`, and `NP-Hard`. This lesson introduces those classes conceptually. You will see how `P` collects problems solvable in polynomial time, how `NP` collects problems whose solutions can be *verified* in polynomial time (even if finding them might be slow), and how `NP-Complete` and `NP-Hard` capture the hardest problems in `NP` and beyond. You will meet famous examples (Traveling Salesman, SAT, the knapsack problem), get a light introduction to **polynomial-time reductions** as the tool used to prove a new problem is `NP-Complete`, and confront the legendary `P vs NP` question along with why it matters for cryptography and the entire structure of computer science. This lesson assumes you are fluent with all the basic asymptotic machinery from **Asymptotic Analysis Fundamentals**, **Big-O Notation (Upper Bound)**, **Time Complexity Analysis Techniques**, **Space Complexity Fundamentals**, and **Little-o and Little-omega Notations**. Complexity classes use those same growth-rate ideas to draw lines between "polynomial" and "exponential" rather than between specific functions like `n^2` and `n log n`. From here you will move into **Amortized Analysis**, shifting from broad problem classification back to the fine-grained per-operation analysis that complex data structures demand.

When a paper claims an algorithm runs in `O(n^2)`, that statement leaves a question wide open: does the algorithm *actually* take quadratic time, or could the bound be loose? **Little-o** and **little-omega** are the notations that close that gap. `f(n) = o(g(n))` says `f` grows *strictly* slower than `g`, never matching its rate, and `f(n) = omega(g(n))` says `f` grows strictly faster. They are the precision tools of theoretical CS. This lesson defines both notations using the formal limit-based criterion: `f(n) = o(g(n))` iff `lim f(n)/g(n) = 0`, and `f(n) = omega(g(n))` iff `lim f(n)/g(n) = infinity`. You will prove and disprove `o` and `omega` claims for common function pairs (`n` vs `n^2`, `log n` vs `n`, `n` vs `n log n`, `2^n` vs `n^k`), use these notations to rank growth rates strictly, and see how the full five-notation system fits together: `O` and `Omega` allow equality, `Theta` requires it, while `o` and `omega` forbid it. This lesson builds on three earlier asymptotic foundations. From **Big-O Notation (Upper Bound)** you have non-strict ceilings; from **Big-Omega Notation (Lower Bound)** you have non-strict floors; and from **Big-Theta Notation (Tight Bound)** you have the exact-rate case. Little-o and little-omega simply require the inequality to be strict, ruling out the matching case that Big-O and Big-Omega permit. With all five notations under your belt, you will be ready for **Complexity Classes (Conceptual Overview)**, where these comparisons get used to separate classes like `P` and `NP`.

How can a search through a million sorted items finish in roughly twenty comparisons? The answer is `log_2(n)`, and once you internalize what that expression really means, an entire family of fast algorithms (binary search, balanced trees, divide-and-conquer) suddenly stops feeling like magic. **Logarithms & Exponentiation** builds your intuition for the math behind `O(log n)`. You will start from a concrete framing (a logarithm answers the question "how many times can I halve `n` before reaching 1?") and work through the powers-of-2 sequence (1, 2, 4, 8, 16, ...) that appears constantly in computer science. You will pick up the three logarithm properties you actually need (`log(a * b) = log(a) + log(b)`, `log(a^n) = n * log(a)`, and `log(1) = 0`), see why the base does not matter inside Big-O, meet fast exponentiation as a preview of `O(log n)` in action, and trace why every halving loop produces logarithmic complexity. In **Big-O Notation (Upper Bound)**, you learned that `O(log n)` sits between `O(1)` and `O(n)` in the hierarchy and is associated with algorithms that halve their work each step. This lesson supplies the mathematical reason that pattern produces a logarithm, so the complexity class stops being something you memorize and starts being something you can derive. Once logarithms feel comfortable, you will be ready for **Mathematical Sequences**, where you will study arithmetic sums, geometric series, and Fibonacci patterns that explain the running times of everything from nested loops to recursive algorithms.

When you stare at merge sort or binary search, the running time is not naturally a simple sum like `n^2/2 + n/2`. It is something more interesting: the cost of a problem of size `n` is built out of the cost of smaller problems plus a little glue. That self-referential structure is captured by a **recurrence relation**, an equation of the form `T(n) = ...` written in terms of `T` on smaller inputs. This lesson teaches you to spot, write, and classify recurrences directly from recursive code. You will identify the two ingredients every recurrence needs (the recursive case and the base case), then translate familiar algorithms into their `T(n)` equations: factorial gives `T(n) = T(n-1) + 1`, binary search gives `T(n) = T(n/2) + 1`, merge sort gives `T(n) = 2 T(n/2) + n`, and naive Fibonacci gives `T(n) = T(n-1) + T(n-2) + 1`. You will also group recurrences into three families (linear, divide-and-conquer, and multi-branch) and recognize the Big-O class each typically produces. This lesson builds on **Big-O Notation (Upper Bound)**, which gave you the language for expressing complexity, and on **Mathematical Sequences**, which gave you experience with arithmetic, geometric, and Fibonacci-style sequences, exactly the patterns you will see when a recurrence is expanded. Writing a recurrence is half the battle; the other half is converting it into a closed-form Big-O. That is the subject of the very next lesson, **Solving Recurrence Relations**.

An algorithm that runs in `O(n)` time but allocates a fresh array of size `n^2` will not fit in memory long before it would have finished computing. Memory is finite, and many algorithms quietly trade it for speed; the only way to make that trade-off visible is to analyze space the same way you analyze time. **Space Complexity Fundamentals** extends the Big-O lens from operation counting to memory usage. You will see why we draw a line between auxiliary space (the extra storage an algorithm allocates) and total space (auxiliary plus the input itself), and you will classify common algorithms as `O(1)` constant, `O(n)` linear, or `O(n^2)` quadratic in space. You will meet the in-place vs out-of-place distinction that interviewers love to probe, and you will study a first space-time trade-off where caching results turns repeated work into stored data. In **Big-O Notation (Upper Bound)**, you learned to read complexity classes from loop structure and to recognize that `O(c * n)` collapses to `O(n)`. The same vocabulary and the same simplification rules apply here, but the resource being counted is bytes of allocated memory rather than units of work. Once you can reason about both axes at once, you will be ready for **Logarithms & Exponentiation**, where you will build the mathematical intuition behind `O(log n)`, the complexity class that powers binary search, balanced trees, and most efficient divide-and-conquer algorithms.

Recognizing `O(n)` or `O(n^2)` on a slide is one thing; staring at unfamiliar code in an interview and confidently announcing its complexity is another. The gap between those two skills is filled by analysis technique, and that technique is what this lesson hands you. **Time Complexity Analysis Techniques** turns Big-O classification into a repeatable workflow you can apply to any snippet. You will analyze single loops with non-standard step sizes, nested loops where the inner bounds depend on the outer variable, sequential blocks that you sum and simplify, `if`/`else` branches where you take the worst case, and the telltale halving-or-multiplying patterns that produce `O(log n)`. Along the way you will see why a triangular loop summing `0 + 1 + 2 + ... + (n-1)` lands at `O(n^2)`, and why constant-bounded loops collapse to `O(1)` no matter how many times they run. In **Big-O Notation (Upper Bound)**, you learned what `O(f(n))` means and how to spot the dominant term in a polynomial. This lesson sharpens that recognition into a procedure: count iterations precisely, multiply for nesting, add for sequencing, and drop everything but the dominant term. Once you can dissect time complexity from real code, you will be ready for **Space Complexity Fundamentals**, where you apply the same Big-O lens to memory usage instead of operation counts.