We investigate microwave background anisotropies in the presence of primordial magnetic fields. We show that a homogeneous field with fixed direction can amplify vector perturbations. We calculate the correlations of $δ T/T$ explicitly and show that a large scale coherent field induces correlations between $a_\ell-1,m$ and $a_\ell+1,m$. We discuss constraints on amplitude and spectrum of a primordial magnetic field imposed by observations of CMB anisotropies.