Is it just a coincidence that the derivative of is also or is there some feature in the function due to which this happens?
Oh goodness no, of course it’s not a coincidence.
The function which is its own derivative is the fundamental idea here, and the logical starting point. This function is the exponential function \exp(z) which we can construct based solely on that defining property, and from this function we can derive the constant e=\exp(1) (as well as its period 2\pi i, which is the reason \pi is so important), and prove various expressions for it, including
\displaystyle e=1+\frac{1}{1!}+\frac{1}{2!}+\frac{1}{3!}+\ldots = \lim_{n \to \infty}\left(1+\frac{1}{n}\right)^n.
So the “compound interest” aspect is the minor side effect, not the heart of the logical structure. It is often the case that history takes a meandering approach to discovery, and not the most efficient or logical route. The fact that Bernoulli stumbled upon e initially as the limit of (1+1/n)^n is incidental.
Let me elaborate a little bit.
You should begin by seeking a function which satisfies f'=f. This is a very natural and very powerful idea, since plenty of differential equations which arise in math, physics and engineering can be solved using such a function (harmonic oscillators, the normal distribution and many more).
Once you hit upon the idea of finding such a function, you can build it as follows. You start with the simplest function
\displaystyle f(x)=1
(You can also start with f(x)=0, but then you can stop immediately because this function is indeed its own derivative, and this is not useful or enlightening).
When you take the derivative of this f you get, unfortunately, 0, which isn’t f itself. What has a derivative of 1? Well, x does, so let’s add it in:
\displaystyle f(x)=1+x
The derivative is now indeed 1, but ouch, our function has changed in the meantime and now has that x term. What has a derivative of x? Well, \frac{x^2}{2} of course. We can add it in and get a function whose derivative is 1+x:
\displaystyle f(x)=1+x+\frac{x^2}{2}
but once again, our new function has the previous function for its derivative, but not itself. Ugh. So we add another term to that has a derivative \frac{x^2}{2}, and obtain
\displaystyle f(x)=1+x+\frac{x^2}{2}+\frac{x^3}{6}
this seems to always get us in trouble, but if we consider the function which adds up all those terms indefinitely, we are saved by the happy circumstance that in an infinite domino chain, every domino has a follower, and that follower will take care of it. In other words, the function
\displaystyle f(x)=1+x+\frac{x^2}{2}+\frac{x^3}{6}+\ldots = \sum_{n=0}^\infty \frac{x^n}{n!}
does exactly what we want: it is its own derivative. Excited, we give it a name: \exp. At this point we have no reason to suspect that it is anything raised to the power of x. It’s just a function, and a damn important one.
We start exploring our new and fascinating function. It is easy to see that it converges for every x, and in fact converges absolutely. We notice that the following rearrangement of terms is now justified:
\displaystyle \sum_{k=0}^\infty \frac{x^k}{k!}\sum_{m=0}^\infty \frac{y^m}{m!} = \sum_{n=0}^\infty \frac{1}{n!}\sum_{k=0}^n \binom{n}{k}x^k y^{n-k} = \sum_{n=0}^\infty \frac{(x+y)^n}{n!}
and it immediately implies the wonderful addition formula
\displaystyle \exp(x)\exp(y)=\exp(x+y)
Because of that, whatever \exp(1) is, we see that \exp(2) is its square, and \exp(3) is its cube, and so on. In fact, also \exp(3/2) has to be the square root of \exp(3), and proceeding in this way we discover that \exp(q) is simply \exp(1)^q for any rational q, and by continuity, \exp(x)=\exp(1)^x for any real x. Therefore, it is a consequence of our original definition that the solution of f'=f happens to be “some number raised to the power x".
What number? Why \exp(1) of course, which we should give a name to, and so we call it e=\exp(1). Looking back at the definition of \exp we find that
\displaystyle e=1+1+\frac{1}{2}+\frac{1}{6}+\ldots+\frac{1}{n!}+\ldots
and it is a standard exercise using the binomial theorem to conclude that also
\displaystyle e=\lim_{n \to \infty}\left(1+\frac{1}{n}\right)^n
Notice how unnatural it would be to start with this definition and work backwards to the function \exp. It’s convoluted and unmotivated, but the logical flow we just followed makes perfect sense.
So overall, again: The function which solves f'=f is the amazing thing here, and by sheer logic we find that it is given by this pleasant, convergent power series, and it follows that it can be viewed as e^x, with the number e defined as an infinite series or as a simple limit, and that’s where e actually comes from. Not, with all due respect, from the value of $1 compounded continuously at 100% interest.