The only thing that matters is how much of your IQ is actually being applied to the real world. I've met people with IQs higher than mine who were so full of themselves, so unwilling to learn anything new that they were, in effect, dumber than me, and I would look down on them.
It's also possible for someone to have a mental illness that makes much of their intelligence go to waste, or be a burden on them, without providing any real merit.