An adiabatic change of a bound state along a closed circuit in the parameter space can induces holonomies not only in the phase of the state, but also in the associated eigenspace and eigenvalue. The former is the well-known Berry phase while the latter, namely the exotic holonomy, is found a decade ago and its origin has not been understood yet. By extending the parameter into the complex number, the correspondence of the exotic holonomies and the degeneracy of the non-Hermitian Hamiltonian, or the exceptional points, is revealed. We show that this explains all the known non-trivial characteristics of the exotic holonomies.