Derivative convergence for iterative equation solvers

When nonlinear equation solvers are applied to parameter-dependent problems, their iterates can be interpreted as functions of these variable parameters. The derivatives (if they exist) of these iterated functions can be recursively evaluated by the forward mode of automatic differentiation. Then one may ask whether and how fast these derivative values converge to the derivative of the implicit solution function, which may be needed for parameter identification, sensitivity studies, or design optimization. It is shown here that derivative convergence is achieved with an R-linear or possibly R-superlinear rate for a large class of memoryless contractions or secant updating methods. For a wider class of multistep contractions, we obtain R-linear convergence of a simplified derivative recurrence, which is more economical and can be easily generalized to higher-order derivatives. We also formulate a constructive criterion for derivative convergence based on the implicit function theorem. All theoretical results are confirmed by numerical experiments on small test examples.