I understand both of you. I do feel like this sometimes, and honestly sometimes I wish it was the case - sometimes I imagine myself as a ghost when I walk through the halls of my college buildings, I pretend no one can see me and I just observe. I also like to walk around buildings I don't have any classes in, just to observe too, like a ghost seeing how the other students live that I don't get to interact with.
But every so often, I am reminded that I exist. People remember things that I tell them, and it shocks me and catches me off guard. How did they remember that? Why would they care to bring it up again?
It's because people do perceive you, they do care about you and they do think of you as a person in their life. This means that they would perceive your absence if you "left". Little moments like that really help me to remember that there are people in the world who appreciate me, who's lives are, even if its an infinitesimal amount, made more fulfilled by our connection. And it's the same for you.
And here is also the hard truth. The reason I feel so disconnected from others is because I do not reciprocate the same care that they give to me. It shocks me that people remember things about me because I don't remember anything about them. If you live your life as if you are disconnected from everyone, it will begin to come true, and then you will be the one realizing how those interpersonal connections made your life better.