an ideal parasite should be able to thrive within the host! then why didnt natural selection lead to the evolution of such totally harmless parasites?

There are no harmless parasites. By definition a parasite harms the host and they can't eveolve to not harm the host. They are taking food or resources from the host (and in extreme cases they are big and heavy and there's a lot of extra weight to carry round) and so the host will seek to destroy them with its immune system or other mechanisms.

Totally harmless parasitism is called commensalism

..and if the two parties benefit each other then it's called mutualism (or sometimes symbiosis, although people have different interpretations of that word!).

If the original question was less about ecological definitions and more about why evolution hasn't lead to organisms living within other (larger) hosts without causing damage, then the answer is that it has - many times.