Um... yes. Extremely.
You have nerves and "flesh strands" holding your eyeballs in at the back, they don't just shake around in your head like in a cartoon. It would actually take a good amount of strength to pull it out, and if you used tweezers, you're more likely to puncture the eyeball before fully detaching it.