What is a derivative? Building a geometric intuition
The world around us is always changing. Cars drive along the highway, we get older, and temperatures fluctuate. Measuring the rate of change is tricky, as it always requires two points, an interval. I can only measure the change between two points. What if we can measure change at a single instant?
The previous post in this series built a graphical understanding of derivatives as a rate of change over arbitrarily small intervals. This article will explore the d notation as a representation of a very small change. I like to think of this d as “delta” or “difference”.
Note: This post assumes that you have read the previous one in the series. If not, feel free to have a quick look before reading on.
We can define \(dx\) as a very small change in \(x\), and \(dy\) a very small change in \(y\). We will also use \(y\) and \(f(x)\) interchangeably; \(y\) is defined as a function of \(x\).
History of the \(d\) notation
The \(d\) notation for derivatives was developed by Gottfried Wilhelm Leibniz in the late 17th century. Leibniz used \(dy\) and \(dx\) to represent very small changes in \(y\) and \(x\).
Let’s say that we have a simple relationship:
\[ y = 2x \]
The derivative of this function would be the rate of change of that function at a single point. If that sounds confusing, go back to part one. Once you have read part one, find the derivative of this function using the rate of change method.
A fancy rate of change
We can also define the derivative of a function as the relation between a small change in \(x\), noted \(dx\), and a small change in \(y\), noted \(dy\). This should remind the reader of the rate of change formula described in part one:
\[ \frac{f(x + h) - f(x)}{h} \]
Looking at this fraction, the numerator computes the change in the value of \(f(x)\) or \(y\), while the denominator is the change in \(x\). The derivative of the function is the ratio of these two as \(h\) becomes very small.
Using the \(d\) notation, we can express this fraction in the following way:
\[ \frac{dy}{dx} \]
The numerator is a small change in the function value \(f(x)\) or \(y\). The denominator is a small change in \(x\).
So far, so good. This is just the rate of change with fancy notation. Now, let’s use this notation to differentiate the function we started with:
\[ y = 2x \]
We now add a very small change to both sides, \(dy\) and \(dx\). The question is, by how much does \(y\) change when \(x\) changes; in other words \(\frac{dy}{dx}\).
\[ \begin{align*} y + dy &= 2(x + dx) \\ y + dy &= 2x + 2dx \\ dy &= 2x + 2dx - y \\ dy &= 2x + 2dx - 2x \quad \text{(substituting $y = 2x$ into the equation)} \\ dy &= 2dx \\ \frac{dy}{dx} &= 2 \quad \text{(dividing both sides by $dx$)} \end{align*} \]
We have just differentiated \(y = 2x\), well done! If you have not done so already, try differentiating \(y = 2x\) using the rate of change method shown in part one.
Getting geometric
The title mentioned “Geometric Intuition” and so far we have just covered mathematical notation and some algebra. We are now getting to the geometry of it. Let’s take another function:
\[ y = x^2 \]
Here, \(y\) can be thought of as the area of the square with sides of length \(x\). The derivative of \(y\) with regards to \(x\) answers the question: how does the area of the square with sides of length \(x\) increase with \(x\)?
Let’s draw this square:
What happens when I increase \(x\) by \(dx\)? Think about it before reading on.
This is shown in the picture below:
As you can see, the total area (\(y + dy\)) is now:
\[ y + dy = x^2 + dx \cdot x + dx \cdot x + dx^2 \]
Each \(dx \cdot x\) represents one of the long rectangles next to the original square.
As we are only interested in the change in area \(dy\), we can subtract the previous area \(x^2\) from the equation and get:
\[ dy = 2x \cdot dx + dx^2 \]
As \(dx\) is very small, much smaller than in the drawing, we can get rid of \(dx^2\), an even smaller number. This leaves us with:
\[ dy = 2x \cdot dx \]
\[ \frac{dy}{dx} = 2x \]
We have successfully differentiated \(y = x^2\), pat on the back.
Some practice
To test your understanding, try differentiating the following functions with the \(d\) notation.
- \(y = x^3\)
- \(y = \frac{1}{x}\)
- \(y = \sqrt{x} + x^2\)
Final Thoughts
This notation is not only notation for the sake of more notation, it is another approach to the problem of derivatives, to understanding how things change. It encourages us to think in small changes, which is critical for more advanced applications of derivatives; such as Machine Learning or Optimisation. It is all about understanding how small changes (\(dx\)) ripple through a system (\(dy\)).
It is one more step in trying to understand the cause of our actions and make sense of the complex reality around us.