On the mathematical foundations of smoothness constraints for the determination of optical flow and for surface reconstruction

Gradient-based approaches to the computation of optical flow often use a minimization technique incorporating a smoothness constraint on the optical flow field. The author derives the most general form of such a smoothness constraint that is quadratic in first derivatives of the grey-level image intensity function based on three simple assumptions about the smoothness constraint: (1) it must be expressed in a form that is independent of the choice of Cartesian coordinate system in the image: (2) it must be positive definite; and (3) it must not couple different component of the optical flow. It is shown that there are essentially only four such constraints; any smoothness constraint satisfying (1), (2), or (3) must be a linear combination of these four, possibly multiplied by certain quantities invariant under a change in the Cartesian coordinate system. Beginning with the three assumptions mentioned above, the author mathematically demonstrates that all best-known smoothness constraints appearing in the literature are special cases of this general form, and, in particular, that the `weight matrix' introduced by H.H. Nagel is essentially (modulo invariant quantities) the only physically plausible such constraint