What makes mathematics beautiful?

What makes anything beautiful?

We don't know exactly, of course, and there's no single answer. But some themes come to mind.

A subtle blend of the familiar, the reassuring, with the surprising, the unexpected.

Many answers here mentioned the aspect of "surprise" or "unexpected connections". Those are beautiful, but they cannot live in isolation of some structure, some cohesiveness. A completely random mess is highly unexpected, but it fails to elicit any sense of beauty.

Random notes or white noise do not make us gasp with appreciation. In all beautiful musical compositions there's a deliberate tension between the places where everyone would correctly guess the next note vs. the twists, the shocks, the startling turns of phrase that send shivers down our spine.

Randomly colored pixels aren't pretty.

But if you blend some order with the chaos, you could be Paul Klee.

The unexpected surprises doled out by mathematics are beautiful only inasmuch as they adorn structure, consistency and integrity.

Complexity and depth.

Simple, shallow things can be pretty, or catchy, even exciting. But this doesn't last long. Works of art that leave us in continuous awe are uniformly multi-layered, deep and complex.

(Chopin's Etude op. 10, no. 11 in E-flat major)

Few things are as deep and complex as mathematics.

The raw intensity of the natural numbers is enriched by increasingly more sophisticated structures - the ring of integers, the fields of rational numbers, the reals, the complex numbers, the deeper layer of abstract rings and fields, ideals, yet another layer of zeta functions and L-series which connect a completely different domain (complex function theory), and Riemann surfaces, and elliptic curves, and étale cohomology, and on and on it goes in seemingly endless variety of extensions, abstractions, distillations and analogies. The breadth and depth are staggering.

Symmetry, and flaws.

Symmetric structures are pretty, and mathematics has them in abundance - in fact, large swaths of math are the study of symmetry.

But it doesn't stop there. Perfect, complete symmetry may get dull after a while. Subtle variations are enforced by structures that are almost, but not quite, symmetric, like aperiodic tilings.

Some of the most powerful ideas of mathematics require subtle violations of symmetry to get off the ground. The complex sphere is more symmetric than the complex plane, but it supports almost no analytic functions. Seen in this way, the complex plane is a brilliant, broken structure: take the perfect sphere, pierce it and unravel the beauty that emerges. The tension between modular forms and modular functions is another instance of the same idea.

The definition of a field, one of the fundamental objects of algebra, has addition and multiplication satisfy almost the same axioms - but only almost. If you force complete symmetry between the two, nothing of interest remains.

The self-similarity of many fractals and dynamical systems is only approximate.

Abstraction and reality

There's a spectrum stretching from completely abstract and ethereal to perfectly physical and natural. Mathematics has domains everywhere on that spectrum, and people find their personal beauty in many different locations on it.

Regular trees are a mathematical abstraction, but there exist real manifestations of them.

Navier-Stokes governs fluid dynamics.

Raw, inexplicable beauty.

Many things just don't fall neatly into any of those themes. There are plenty others I'm forgetting or unsure how to describe.