That's the notion of "rights" we have in the US though. It's the same with the Bill of Rights. It's true some states do go further and bestow more affirmative rights. But it's deeply ingrained in US political thought that "right to do X" means "government won't stop you from doing X", not "government will stop anyone who tries to stop you from doing X".