I don't think that you are weird for worrying about this. It's natural for people to want others to think the best about them and to worry what others will think of them .
When I was younger, someone told me to stop worrying about what others think about you and just live your life. They pointed out that you can't control what other people think. Whether they are right or wrong about something, they are going to think whatever they want to think, and you can't change that.
I know it is easier said than done, but you should try to worry less what other people think. I have worried somewhat what other people might think or say when they hear that I ctb. Sometime I still worry, but decided that it really doesn't matter, because I will be dead, and whatever they say can't hurt or affect me anymore. Even if some people gossip or say something unkind, it won't be for very long. They will move on and start gossiping about something else very quickly. They always do.
Generally speaking, people usually only talk about the good times and good qualities of people when they remember someone who has died.