Data too large to fit into memory

Big data

Data too large to be loaded into main memory--the information must reside on disk (or come as stream) and be brought into main memory selectively for processing.
This note is to give you a brief idea to the data structures & algorithms for working with big data, using simple applications.

(Most) Data stay in the disk --- Disk-based applications

Sinopsis data structures

Synopsis data structures are any data structures that are substantively smaller than their base data sets.
An $f(n)$ synopsis data structure for a class $Q$ of queries is a data structure for providing (exact or approximate) answers to queries from $Q$ that uses $O(f(n))$ space for a data set of size $n$ where $f(n) = o(f(n^\epsilon))$ for some constant $\epsilon < 1$.

Streaming algorithms