Why do we keep searching for exponential solutions? Do we just pull them out of a hat?

Invariance of the operator in "translations''

One way to understand why exponential functions keep appearing is because of symmetries under translations that the operator in question may have. And when we say "translations" here we mean both spatial or temporal translations, i.e., invariance under $\boldsymbol{x}\mapsto\boldsymbol{x}+\boldsymbol{a}$ or $t\mapsto t+\tau$ for any $\boldsymbol{a}$ or $\tau$.

That is, exponential functions appear generically as a consequence of the invariance of our equations under some direction.

To see that we'll start with a basic theorem for commuting operators.


Exercise 1: Show that when two operators $\mathbb{A}$ and $\mathbb{B}$ commute then they share the same eigenvectors/eigenfunctions. (Note, this does not mean that they share the same eigenvalues necessarily.)

(Two operators that act on vector space $\mathcal{H}$ commute when $\mathbb{A}(\mathbb{B}\,u) = \mathbb{B}(\mathbb{A}\,u)$ for every $u\in\mathcal{H}$.)

The spectrum of an operator is the set of all its eigenvalues. An eigenvalue is degenerate when there are multiple eigenvectors that correspond to the same eigenvalue. For example, if $\mathbb{A}$ is an operator acting on space $\mathcal{H}$ and if:

$$ \mathbb{A}\, u_1 = \lambda\,u_1,\ \textrm{and}\ \mathbb{A}\, u_2 = \lambda\,u_2, $$

where $\lambda\in\mathbb{C}$ and $u_1, u_2\in \mathcal{H}$ then eigenvalue $\lambda$ is degenerate.

It's easy to show that two commuting operators share the same eigenvectors. Consider first the case where operator $\mathbb{A}$ is not degenerate, i.e., each of every eigenvalue $\lambda_j$ of $\mathbb{A}$ corresponds to a single eigenvector $u_j$. Then show that vector $(\mathbb{B}u_j)$ is also an eigenvector of $\mathbb{A}$ with eigenvalue $\lambda_j$. That is, show that

$$ \mathbb{A}\, (\mathbb{B}u_j) = \lambda_j\,(\mathbb{B}u_j). $$

But we said that $\mathbb{A}$ is not degenerate, thus eigenvalue $\lambda_j$ only has one eigenvector. This means that $u_j$ and $(\mathbb{B}u_j)$ must be parallel, i.e., multiples of one another. (Remember that an eigenvector is determined up to a normalization constant). But if $u_j$ and $(\mathbb{B}u_j)$ are paralled we are done since then

$$ \mathbb{B}\, u_j = \mu u_j, $$

which proves the initial assertion.


Nothing too fancy right? But wait.. we've only proved it for when $\mathbb{A}$ is not degenerate. Actually we can carry on, more or less, the same proof when $\mathbb{A}$ is degenerate but with some additional extra technicalities. Let's not go into those details for the moment -- you can refer to any Linear Algebra book for that or, e.g., to Shankar's chapter 1.

Why is this useful?

We can show that when we have a linear operator $A$ which is invariant under some translation then $A$ commutes with the translation operator.

Specifically, the translation operator by $a$, e.g., in $x$, is defined through its action on functions $\phi(x)$:

$$ \mathbb{T}_a\,\phi(x) = \phi(x+a). $$

Say, for example, that we want to solve the eigenvalue problem

$$ A\,\phi(x) = \sigma\,\phi(x),\tag{1} $$

where operator $A$ is some operator that is invariant under translations $x\mapsto x+a$. Then, equation (1) implies that if $\phi(x)$ is a solution of the eigenvalue problem above, then so is $\phi(x+a)$ (since under $x\mapsto x+a$, $A$ remains unchanged and $\phi(x)\mapsto\phi(x+a)$). In other words,

$$ A\,\phi(x+a) = \sigma\,\phi(x+a). $$

But from the left-hand-side of the above, we have $$ A\,\phi(x+a) = A\big(\mathbb{T}_a\,\phi(x)\big),\tag{2} $$ while from the right-hand-side, $$ \sigma\,\phi(x+a) = \sigma \mathbb{T}_a\,\phi(x) = \mathbb{T}_a\big(\sigma\,\phi(x)\big) = \mathbb{T}_a \big(A\,\phi(x)\big).\tag{3} $$ For the last part of (3) we used (1).

Equations (2) and (3) imply that $A$ and $\mathbb{T}_a$ commute. Therefore, by invoking Exercise 1 we know that the eigenfunctions in equation (1) are nothing else from the eigenfuctions of $\mathbb{T}_a$.


Exercise 2: Show that the eigenvectors/eigenfunctions of the translation operator are single harmonics.

The translation operator by $a$ that acts on function $\phi(x)$ is defined through:

$$ \mathbb{T}_a\,\phi(x) = \phi(x+a) $$

(Note that there can be two definitions; some define it as $\mathbb{T}_a\,\phi(x) = \phi(x-a)$. The difference lies in that the first definition describes a so-called "passive" transformation, i.e., a translation of the reference frame, while the second definition an "active" transformation, i.e. a translation of the actual field $\phi$.)

To find the eigenfunctions of $\mathbb{T}_{a}$ we will first show that $\mathbb{T}_{a}$ commutes with a differrent operator (whose eigenvectors are easier to find) and then use Exercise 1.

Show that $\mathbb{T}_a$ commutes with the differentiation operator with respect to $x$. That is show that for every $\phi(x)$ we have that:

$$ \frac{\mathrm{d}}{\mathrm{d}x} \big(\mathbb{T}_a\,\phi(x)\big) = \mathbb{T}_a\Big(\frac{\mathrm{d}}{\mathrm{d}x}\phi(x)\Big). $$

Next find the eigenfunctions of the differentiation operator, i.e., find functions $u(x)$ that satisfy:

$$ \frac{\mathrm{d}}{\mathrm{d}x} u(x) = \lambda\,u(x), $$

which implies that $u = e^{\lambda x}$. Lastly, by using Exercise 1 we have that exponentials $e^{\lambda x}$ are also eigenfunctions of $\mathbb{T}_a$.


(At this point we cannot restrict the eigenvalue $\lambda$ of the $\mathrm{d}/\mathrm{d}x$ operator. To do so we need invoke some more properties of the vector space that the operator acts on. For example, a usual scenario in physics is to take as the vector space that functions $\phi(x)$ live in the Hilbert space $\mathcal{H}$ that consists of square-integrable functions which additionaly remain bounded at $|x|\to\infty$.)


Exercise 3 Consider the Hilbert space $\mathcal{H}$ defined above and show that the eigenvalues $\lambda$ of the translation operator $\mathrm{d}/\mathrm{d}x$ have to be purely imaginary. What does that imply for the eigenfunctions of $\mathrm{d}/\mathrm{d}x$ and, in turn, for the eigenfunctions of $\mathbb{T}_a$. All we are left to do then is to find the eigenfunctions of translation operator $\mathbb{T}_a$.


A plausible objection to the above

"But isn't this again, yet another trick? Could we have solve the eigenproblem of the translation operator without having to resort in figuring out that $\mathbb{T}_a$ commutes with the differentiation operator $\mathrm{d}/\mathrm{d}x$?

The answer is yes.

So, say we want to find functions $u(x)$ that satisfy the eigenproblem

$$ \mathbb{T}_a\,u(x) = \lambda(a)\, u(x) $$

We have included a dependance of the eigenvalue in $a$. In general, the eigenvalues of the translation operator may differ with the amount of translation $a$. So which functions satisfy:

$$ u(x+a) = \lambda(a)\, u(x) $$

Remember that eigenfunctions $u$ can be determined up to a constant. Say that we pick the constant so that $u(0)=1$. Then, by evaluating the above at $x=0$, we have that $\lambda(a) = u(a)$ which implies:

$$ u(x+a) = u(a)\, u(x)\tag{*} $$

The above holds for every $a$. Assume an infinitesimal translation $a\ll 1$ which allows us to write $u(a)\approx 1 + c a$, where $c$ is a constant (actually, $c$ it's just $u'(0)$). Then, if $x=n a$ the function $u(x)$ is nothing else than

$$ u(x) = \lim_{n\to\infty} u(na) = \lim_{n\to\infty} \big[u(a)\big]^n \approx \lim_{n\to\infty} \big[ 1 + c \frac{x}{n}\big]^n = e^{c x} $$

Voilá! The exponential appears once again.

One could also avoid thinking of $x=na$. From ($*$) and for $a\ll 1$ by expanding both sides we get: $$ u(x) + a u'(x) = \big[ 1 + a u'(0)\big] u(x) $$ which simplifies to $u'(x) = u'(0) u(x)$ with solution $u(x) = e^{u'(0)\,x}$.

Now if we plug it back in $\mathbb{T}_a\,u(x) = \lambda(a)\, u(x)$ we can also determine the eigenvalue, $\lambda(a) = e^{ca}$. At this point $c$ can be anything. But if, for example, we restrict ourselves to functions that remain bounded at $|x|\to\infty$ then $c$ must be purely imaginary.

But then why did we even mention $\mathrm{d}/\mathrm{d}x$ in the first place?

The translation operator $\mathbb{T}_a$ and the differentiation $\mathrm{d}/\mathrm{d}x$ are intimetely related. Just consider the translation operator for $a\ll 1$ and use Taylor's expansion:

$$ \mathbb{T}_a\,\phi(x) = \phi(x+a) \approx \phi(x) + a \frac{\mathrm{d}}{\mathrm{d}x}\phi(x), $$

which implies that $\mathbb{T}_a = 1 + a \frac{\mathrm{d}}{\mathrm{d}x}$ for $a\ll1$. No wonder they commute!

Actually, using Taylor's expansion we can express $\mathbb{T}_a$ for any $a$:

$$ \mathbb{T}_a\,\phi(x) = \phi(x+a) = \sum_{n=0}^\infty \frac1{n!}\Big(a^n \frac{\mathrm{d}^n}{\mathrm{d}x^n}\phi(x)\Big), $$

or

$$ \mathbb{T}_a = \sum_{n=0}^\infty \frac1{n!}\Big(a \frac{\mathrm{d}}{\mathrm{d}x}\Big)^n. $$

But this is nothing else from the definition of the exponential function:

$$ \mathbb{T}_a = \exp\Big({a \frac{\mathrm{d}}{\mathrm{d}x}}\Big). $$

We say that $\mathrm{d}/\mathrm{d}x$ is the generator of translations $\mathbb{T}_a$.

References

In my opinion the best concise and intuitive summary of linear algebra, vector spaces and Hilbert spaces is found in chapter 1 of:

  • Shankar, Ramamurti. Principles of quantum mechanics. Springer Science & Business Media, 2012.

For the non-physisists this may sound intimitating and also counter-intuitive. Nevertheless Shankar devotes the first chapter of his book solely in linear algebra since it turns out to be a necessary part of quantum mechanics.