linear thinking

I feel I have always been taught just enough math to get by. Since I am by training a physicist and work in the area of medical imaging, that means that I have digested some amount of math, but I have come to realize that what I know mostly amounts to tricks, and when I step back and try to see whether those tricks form a coherent whole, I am sadly disappointed.

Part of the problem may be that the same mathematical concepts crop up sometimes in two or more analogous contexts but at other times they get used in contexts that are fundamentally different. Take, for example, the supposedly simple concept of "linearity." When this term is applied to a function, e.g. height = some function of distance h(x), it very simply means that the function describes a ramp of constant slope. If you figure out the height you reach by travelling a horizontal distance x₁ and then go back to your starting position and figure out the height for the distance x₂, then the total height you achieve by travelling x₁ followed by x₂ is simply the linear sum of the two separate heights. This implies that h(x) = c₁ x + c₂, where c₁ and c₂ are constants. The word linear here ends up meaning a function of x that doesn't curve, i.e. doesn't have terms like x².

How about moving across a terrain that isn't just going up a hill with a constant slope. Let's make the height some function h₁(x) that goes up and down like a piece of rolling countryside. Now we imagine being in some airplane that is designed to change its height relative to the ground h₂(x) which could for example be the horizontal distance it travels squared, in which case h₂(x)= x². In general the total height is given by: h(x) = h₁(x) + h₂(x), which is no longer a linear function of distance. And indeed, if you repeat the same experiment by going separate distances x₁ and x₂, the heights no longer add up to the overall height you achieve by travelling x₁ + x₂.

Let's say you are able to make adjustments to your plane so that the height reached is given by: h(x) = h₁(x) + c h₂(x). Here, the plane still rises above the ground in a manner described by h₂(x), but the parameter c is a multiplier that determines the scale of this effect. The point is that this system isn't linear in how it behaves as a function of distance, but it is linear with respect to the combining of the height of the ground plus the additional height achieved by the plane.

So what, you may well ask. Well, the point is that if you know the height of the countryside, in this case h₁(x), and you know how the plane rises above this height as a function of x, given by h₂(x), then the measured height of the plane as a function of x, h(x) enables you to solve for c, because this physical system is described by a linear system of equations. The final height of the plane has to be a linear combination of the two functions of x, h₁(x) and h₂(x). It turns out to be a simple matter to figure out what proportion of each function is needed to achieve that height.

This is an example of linear regression. Solving for the magnitude of a constant slope hill is another example of linear regression, but in this case the fact that the function of x is itself linear is in fact irrelevant. What is relevant is that the height (measured relative to some arbitrary height such as the sea level) is the linear combination of a constant term and another term (i.e. function of x) which just happens to be linear.

In a follow-up post I will talk more about linear systems, before moving on to the concept of non-linearity. By that stage, what might appear here to be nitpicky hair-splitting becomes genuinely important to actually understanding what is going on.