How do I solve this

Whatever function f(x) is represented by the infinite nested radical

\displaystyle f(x)=\sqrt{x+\sqrt{x+\sqrt{x+\cdots}}}

we can reasonably expect it to satisfy

\displaystyle \sqrt{x+f(x)}=f(x)

This is because what follows the first “x+” in the nested radical is precisely the same nested radical.

The functional equation can be squared to yield

x+f(x)=f(x)^2

which is a quadratic equation in the unknown function f, and can be solved just like any quadratic equation. We find:

\displaystyle f(x)=\frac{1}{2}\left(1\pm\sqrt{1+4x}\right)

At this point we may be unsure about the correct sign to pick for that \pm thingie. It may even be a different choice for different values of x. Notice, however, that for any 0<x\le 1, the sequence

\displaystyle \sqrt{x},\sqrt{x+\sqrt{x}},\sqrt{x+\sqrt{x+\sqrt{x}}},\ldots

is monotonically increasing and bounded (by, say, 2), so it converges, and its limit is positive. Therefore we conclude that

\displaystyle f(x)=\frac{1}{2}\left(1+\sqrt{1+4x}\right)

(This doesn’t work at x=0, since then the sequence is not monotonically increasing and simply stays put. But the value at a single point doesn’t affect the integral.)

Finally, the integral is elementary:

\displaystyle \int_0^1 f(x)\,\mathrm{d}x=\frac{5}{12}\left(1+\sqrt{5}\right)