Implement deriv and fit_deriv for compound models #3783

Compound models currently do not implement analytic derivatives (with respect to their parameters) through a fit_deriv method. This hasn't been a showstopper, since all of the optimizers that use the derivative can use an estimate if an analytic expression is not provided.

However, this is much less straightforward than mere evaluation of compound models. It will have compute partial derivatives with respect to each parameter in the compound model, and will have to take into account which operators each model is combined with, and implement the appropriate product rule, quotient rule, chain rule, etc. There's nothing especially complicated about the math here, but getting the implementation right may be a bit tedious. The most annoying part about this is that we'll have to implement derivatives with respect to the inputs for each model (fit_deriv implements partial derivatives w.r.t. the fittable parameters). Perhaps there should be a separate issue for this.

To provide some hints for anyone who might like to tackle this, model evaluation is currently implemented by passing a dictionary mapping operators (given as strings) to functions that implement that operator between two models:

For example, the '+' operator is simply implemented as "A(x) + B(x)" where A and B are the evaluate methods of two models (there is a little more complexity involved to support things like multiple inputs/outputs. You can see how this is implemented in _make_arithmetic_operator above.

Similarly the '|' operator is implemented to evaluate "B(A(x))" (again modulo some slight complications for generalization, and parameter handling--let me know if these need better explanation).

So for evaluating fit_deriv, if you have a model like "A + B", the '+' operation would have to be implemented like A.fit_deriv(x) + B.fit_deriv(x). That's easy enough. But for implementing '|' we'll need a derivative method. So "A | B" could be implemented like "B.deriv(A(x)) * A.fit_deriv(x)". Though things get more complicated for models of multiple arguments, etc. Not also that in this example, if "B" is itself a compound model it will have to use a B.deriv for the entire compound model; i.e. this would have to work recursively.