Sun worship, veneration of the sun or a representation of the sun as a deity, as in Atonism in Egypt in the 14th century BCE. Though almost every culture uses solar motifs, only a relatively few cultures (Egyptian, Indo-European, and Meso-American) developed solar religions.
www.britannica.com
Sun worship appears to have been very common. After all, all life is ultimately driven by the sun.
Prehistoric cultures were not dumb monkeys with spears - by the time hominids developed enough ingenuity to create technology, they most likely already had a highly complex understanding of their surroundings, including celestial phenomena.
We can see this from many prehistoric monuments like Stonehenge and Newgrange, where the builders had a clear understanding of generational precession of the stars.
Their understanding may not have been scientifically accurate by today's standards, but intuitive understanding can be every bit as comprehensive.
Did people also fear that which they may have worshiped? Do we today?
Worship or fear of such things may have been exactly the same as more modern society regards such things. After all, the time difference between early modern humans and today is negligible in terms of our evolution (though our cognitive advancement has accelerated hugely due to technological power creep).
So I've no doubt there would have been an elite caste of the religious who used alleged 'control' of, or intercession with, natural phenomena like the sun in order to lord it over the masses.
After all, our structures of power and prestige today had foundations in the patterns of prehistory.
So I doubt anyone looked up at the sun, that they'd lived with all their lives, and though "Shit, it's coming to get me, Raaaaaaaaaaa!"
But they probably did have a certain respect/fear of the elite caste who may have appropriated power from the claimed control of natural phenomena.
Though who knows, maybe cavemen spent all day running away from the sun in fear.
I mean, they were 'CAVEmen' right?
Edit: a TLDR in answer to you original titular question: Yes.