We report on the fast excitation of a single atom coupled to an optical cavity using laser pulses that are much shorter than all other relevant processes. This creates a single photon source combining the conceptual simplicity of experiments on free-space atoms with the efficiency and directionality of sources based on cavity quantum electrodynamics. The cavity frequency further constitutes a parameter to design the emitted single-photon wave packets, for example, in a superposition of two tunable frequencies.