I understand the statement in terms of- fear can hold us back from doing things that may eventually bring us joy. It can potentially hold us back from even trying.
I feel the same though. I've kind of accepted now that it's all a compromise. Either you face the things that terrify you. Potentially making yourself more unhappy short-term in the hopes that long-term, it will work out for the better- and it doesn't always! Or, you accept that your fear is going to likely keep you in an unhappy but at least, safer feeling place.
I've definitely taken this route with social anxieties. I'd rather not be around people, even if that means I may feel lonely from time to time. Even if it means I haven't progressed as far in my career as I would have liked. It still feels preferable.
I think you can also feel a sense of peace from feeling like you aren't going to push yourself to do things you find truly awful anymore. That in itself can bring some sense of relief/happiness.