The derivative tells us something about the rate at which a function f increases or decreases at some point x. This rate is the slope of the tangent line to the graph of f at x. So, the derivative tells us something about the “linear” behavior of a function. However, this does not tell us anything about its curvature. To visualize the issue, compare the behavior patterns of functions f(x) = x2 and g(x) = −x2 for x = 0. Since x2 ≥ 0, it is obvious that x = 0 is a minimizer of f and a maximizer of g. This is a stationarity point, as it is easily checked by finding the derivatives of the two functions:
The slope of f is negative for x < 0 and positive for x > 0. This implies that the function is decreasing for x < 0 and increasing for x > 0, but this in turn implies that x = 0 is a minimum. The pattern for g is just the opposite one. In the case of f, the slope itself is increasing, whereas it is decreasing (turning from positive to negative) for g. We can see this by taking the derivatives of f′ and g′:
Here we have used f″(x) to denote the derivative of the derivative. This is referred to as the second-order derivative, whereas the derivative that we have seen so far is actually called first-order derivative.
DEFINITION 2.11 (Higher-order derivatives) Given a continuously differentiable function f, its second-order derivative at point x is the derivative of the first-order derivative. This is denoted as f″(x) or
Taking the derivative of the second-order derivative, we get the third-order derivative:
More generally, we define the k-th order derivative of f as
In the definition, we use the term continuously differentiable. We recall that this simply means that the function is differentiable and its derivative is a continuous function; otherwise, we could not take the derivative of the derivative.
Example 2.30 Given the polynomial function f(x) = 3x3 − 2x2 + 5x − 10, we have
We see that a polynomial of degree n has zero derivative from order n + 1 on.
It is worth noting that for a linear function f(x) = a + bx, we have f″(x) = 0; in fact, a linear function has no curvature. For nonlinear functions, knowing the kind of curvature helps us in plotting them more accurately, and this information can be exploited to find their maxima and minima. In fact, first- and second-order derivatives can be used to characterize the local behavior of a function near some point x0. It is natural to wonder if we could exploit knowledge about higher-order derivatives, provided they exist, to shed some more light on the function. Indeed, it can be shown that by using the derivatives of a function near some point x0, we can find an arbitrarily good approximation of the function by a polynomial, subject to some assumptions, including continuity. We will not state the theorem exactly, as it would require a few technicalities; nevertheless, the result is quite useful, as polynomial functions are relatively easy to deal with.
DEFINITION 2.12 (Taylor’s expansion) Taylor’s expansion of order n for function f around point x0 is given by the following approximation:
A few comments are in order:
- The formula relies on the existence of all the derivatives involved.
- The formula above tells us how we can approximate function f by a polynomial, for a small displacement h around x0.
- The quality of the approximation improves if we increase the order of the involved derivatives.
- On the contrary, the quality of the approximation worsens if we increase the displacement h.
Actually, the formula of Taylor’s expansion relies on a theorem that states that the function can be expressed by the formula above plus a remainder term. This remainder depends on derivative of order n + 1 evaluated at some point in the neighborhood of x0. In practice, this remainder is negligible if we do not get “too far” from x0.
It is quite useful to consider in more detail Taylor’s expansion if we stop with the first- or second-order derivative. If we stop with first-order derivative, we get the first-order Taylor expansion, which can be equivalently rewritten by setting h = x − x0:
We immediately see that the first-order approximation is equivalent to a finite-difference approximation of the first-order derivative:
Formally, this finite-difference approximation can also be expressed as
Table 2.2 Accuracy of first- and second-order approximations of the exponential function.
to remind us that we are substituting infinitesimal increments with finite ones. Furthermore, we see that (2.14) approximates the function by its tangent line;18 hence, it is a linear or first-order approximation. If we want to keep curvature information in the approximation, we can include a quadratic term from Taylor’s expansion:
This is a quadratic approximation around x0; the second-order Taylor expansion is just a parabola.
Example 2.31 To illustrate Taylor’s expansions, we can prove Eq. (2.6), which gives a concrete way to evaluate an exponential function with base e. The exponential function ex is a peculiar one, as f(n)(x) = ex for any n. Its Taylor’s expansion around x0 is then
Now, if we set x0 = 0 and let n → +∞, we get
To check the accuracy of this expansion, let us compare the true value of ex against the first- and second-order approximations f1(x) = 1 + x, and f2(x) = 1 + x + x2/2. The numerical results are displayed in Table 2.2; in Fig. 2.22 we may also visually compare the exponential function with its first- and second-order approximations around x0 = 0.
As expected, low-order Taylor’s approximations deteriorate rather quickly when we depart from the point at which they are developed. Indeed, practical numerical approximations often rely on more sophisticated approaches. Nevertheless, Taylor’s expansions are both conceptually and practically relevant. One fundamental application is sensitivity analysis.
Fig. 2.22 The exponential function (dashed line) against its first- and second-order Taylor expansions.
Leave a Reply