Why don't we just invent math that will allow us to solve problems like x^x = 2 so we don't have to use numerical methods?

We did. But there’s plenty of confusion around what it means to “solve” equations.

Take x^3=2. Can you solve it? Sure: x=\sqrt[3]{2}. So we have the “math” to solve this equation without resorting to numerical methods. Right…?

Well, what is that \sqrt[3]{\cdot} thing? It’s simply, by definition, that function which solves x^3=a. We just gave it a name. How is that “a solution”?

See, we could do the exact same thing with x^x=a. For any a>1 there’s a unique number x such that x^x=a, so we can define a function Z such that x=Z(a), and there you go – we’ve invented the math required to solve the equation. The solution of x^x=2 is x=Z(2).

You’re probably finding this unsatisfactory, but it’s crucial to understand that there’s nothing about the function Z that’s fundamentally inferior to the functions \sqrt[3]{\cdot} or \arccos(\cdot) or any other named function you’ve encountered.

If you wish to determine the numerical value of Z(2) you’ll have some work to do, and it’s “numerical”. But the exact same thing is true of \sqrt[3]{2} or \log(2).

Mathematicians did spend way more time with the exponential, logarithmic and trigonometric functions than they did with the function Z, simply because the equations they come from are tremendously more important. But we can develop series representations and various other formulas for Z just like we did for those other functions.

We actually created a whole lot of math to deal with solving equations. The process we just looked at is one of inversion: you have some function y=f(x) and you wish to invert it to produce x=f^{-1}(y). How and when this can be done is the subject matter of the inverse function theorem and the implicit function theorem, two basic results of differential calculus which have many variants and extensions.