Cellular automata are used as simple mathematical models to investigate self-organization in statistical mechanics. A detailed analysis is given of "elementary" cellular automata consisting of a sequence of sites with values 0 or 1 on a line; with each site evolving deterministically in discrete time steps according to definite rules involving the values of its nearest neighbors. With simple initial configurations; the cellular automata either tend to homogeneous states; or generate self-similar patterns with fractal dimensions ≃ 1.59 or ≃ 1.69. With "random" initial configurations; the irreversible character of the cellular automaton evolution leads to several self-organization phenomena. Statistical properties of the structures generated are found to lie in two universality classes; independent of the details of the initial state or the cellular automaton rules. More complicated cellular automata are briefly considered; and connections with dynamical systems theory and the formal theory of computation are discussed.