We report on optomechanical coupling in ultra-high Q crystalline whispering-gallery-mode resonators. The high purity of the material enables optical quality factors in excess of 10^10. Optical displacement monitoring at the 10^-18m/Hz^1/2-level reveals mechanical radial modes at frequencies up to 20 MHz, corresponding to unprecedented sideband factors (>100). Simultaneously, mechanical quality factors exceeding 10^5 are attained, limited by clamping losses. The fabrication of CaF2 microdisc resonators allowed the observation of several orders of radial breathing modes with low effective mass. Together with the weak intrinsic mechanical damping in crystalline materials, this renders crystalline WGM micro-resonators a promising platform for optomechanical experiments in both the classical and quantum regime.