Aside from a minority of highly ignorant people who equate atheism with devil worship (or the like), there are a worryingly large number of normal people who see atheism as just another one of the vast array of often conflicting religions on offer on this planet. I'd just like to make it clear that atheism is not a religion; it's not even accurate to describe it as a movement. I'll try to explain...
The Oxford English Dictionary* defines religion to be:
noun
1. The belief in and worship of a superhuman controlling power, especially a personal God or gods.
2. A particular system of faith and worship.
3. A pursuit or interest followed with devotion.
— ORIGIN originally in the sense life under monastic vows: from Latin religio ‘obligation, reverence’.
And atheism to be:
noun
The belief that God does not exist.
— DERIVATIVES atheist noun atheistic adjective atheistical adjective.
— ORIGIN from Greek a- ‘without’ + theos ‘god’.
I'll address each of the three definitions of religion with reference to that of atheism:
1. The belief in and worship of a superhuman controlling power, especially a personal God or gods
This is self-evidently at odds with the definition of atheism: One is almost the anti-statement of the other, as atheists do not believe in or worship any superhuman controlling power, whether or not that would be a personal God or gods. I have been told that the very act of believing that there is no God means that atheism is a religion because it involves belief. Nonsense! Whilst I definitely disbelieve in any god that I have been made aware of so far, I just as fervently disbelieve in the existence of entities such as the Easter bunny, the tooth fairy and the monster under my bed. Do these disbeliefs count as religions as well? If not, what's the difference?
2. A particular system of faith and worship
This seems self-evident to me, but I've had a lot of people who follow particular religions cite this one as being evidence that atheism is a religion so I'll talk about it a bit more.
Atheists, by the definition of the word, share one thing and one thing only: They do not believe in a God, gods, deities, or any other superhuman, supernatural entities. That is the only thing that all atheists have in common. Of course, two or more individual atheists may be in agreement with regards to world view, moral values, political inclination, sexual orientation, favourite colour and whether or not they're a morning person, but this is categorically nothing to do with the fact that they are atheists. Atheism is not a club that dictates thoughts or viewpoints; there are no rites or rituals, there is no system to follow, and you can't be 'thrown out' for not following any piece of doctrine because there simply is no doctrine to follow.
Atheists don't believe in gods: simple.
3. A pursuit or interest followed with devotion
Yes, many atheists are particularly devoted to their non-belief in supernatural entities and in this respect could be described as 'religious'. But many people follow many and varied aspects of their lives with equal (if not superior) levels of devotion: work, study, hobbies, political matters, even the progress of television shows. If an atheist is to be deemed religious on these grounds, then so must anybody who is devoted to their job, works tirelessly for academic achievement, campaigns for their political party, or puts in the hours to improve themselves at sport, music or any one of an endless number of other activities.
* A standard reference in the cause of semantic arguments, I think!