8. Numerical Derivative Approximator — Theory
Numerical differentiation estimates \(f'(x_0)\) from nearby values of \(f\) using finite differences.
It is essential when you only have data points or when symbolic differentiation is inconvenient.
1) Why numerical derivatives work
The key idea comes from Taylor expansion around \(x_0\):
\[
f(x_0+h)=f(x_0)+f'(x_0)h+\frac{f''(x_0)}{2}h^2+\frac{f^{(3)}(x_0)}{6}h^3+\cdots
\]
By combining values like \(f(x_0+h)\), \(f(x_0-h)\), you can cancel terms and isolate \(f'(x_0)\).
2) Main finite difference formulas
Forward difference (first-order):
\[
f'(x_0)\approx \frac{f(x_0+h)-f(x_0)}{h}
\]
Error (truncation) is \(O(h)\). It uses points to the right of \(x_0\).
Backward difference (first-order):
\[
f'(x_0)\approx \frac{f(x_0)-f(x_0-h)}{h}
\]
Error is also \(O(h)\). It uses points to the left of \(x_0\).
Central difference (second-order):
\[
f'(x_0)\approx \frac{f(x_0+h)-f(x_0-h)}{2h}
\]
Error is \(O(h^2)\) — typically much more accurate than forward/backward for smooth functions.
3) Choosing the step size \(h\)
Two competing effects matter:
- Truncation error decreases when \(h\) decreases (because Taylor higher-order terms shrink).
- Round-off / noise error increases when \(h\) is too small (subtraction of nearly equal numbers + measurement noise).
So there is usually a “sweet spot” for \(h\). This calculator offers an Auto choose \(h\) helper in function mode,
which scans a range and chooses a step that looks stable under Richardson refinement.
4) Richardson extrapolation (error reduction)
Suppose your method has error like:
\[
D(h)=f'(x_0)+C h^p + \cdots
\]
where \(p=1\) for forward/backward, and \(p=2\) for central.
If you also compute \(D(h/2)\), you can cancel the dominant error term:
\[
D_R = D\!\left(\frac{h}{2}\right) + \frac{D\!\left(\frac{h}{2}\right)-D(h)}{2^p-1}
\]
This is often dramatically more accurate (when the function is smooth and values are reliable).
A practical error estimate is:
\[
\text{Estimated error} \approx \left|D_R - D\!\left(\frac{h}{2}\right)\right|
\]
Data warning: for discrete data, \(h/2\) might not exist in your dataset — Richardson will only work if the needed half-step points are present.
5) Differentiation from data points
If you have data \((x_i,y_i)\), you can apply the same formulas by treating \(y_i\approx f(x_i)\).
For example, with central difference you need points at \(x_0-h\) and \(x_0+h\).
If the data is noisy, finite differences can amplify noise. A simple remedy is a local linear fit:
choose the \(k\) nearest points to \(x_0\), fit \(y\approx mx+b\) by least squares, and take \(m\) as the derivative estimate.
6) Worked example (the sample in the calculator)
Given data points for \(f(x)=\sin x\):
\((0,0), (0.1,0.09983), (0.2,0.19867)\), estimate \(f'(0.1)\).
Central difference with \(h=0.1\):
\[
f'(0.1)\approx \frac{f(0.2)-f(0.0)}{2(0.1)}
=\frac{0.19867-0}{0.2}
\approx 0.99335
\]
The true derivative is \(\cos(0.1)\approx 0.99500\), so this is already close.
If you also had half-step data at \(0.05\) and \(0.15\), Richardson could improve accuracy further.
7) Practical tips
- Prefer central difference when you can (smooth functions/data on both sides of \(x_0\)).
- Use forward/backward near boundaries (if you only have one-sided data).
- If your values are noisy, enable local linear fit instead of tiny \(h\).
- If you have a function, try Auto choose \(h\) + Richardson for a stable estimate.