It seems that people have a hard time understanding that even if something doesn't exist a can negatively affect you if other people believe it.

Yes there isn't a magical fairy that gives you gold under your pillow if you stab someone but if 400 people out of the 600 people in your town believe that all of a sudden everyone's getting stabbed.

Same thing applies to religion and other such matters which is why it needs to be contained or it needs to be positive and beneficial for your culture. It's also applies to atheism too because Atheism is technically a belief of lack of belief so it isn't exempt from this.