Religion relies on assumptions that cannot be proven physically, secularism just avoids those assumptions (I say this as a Christian who finds secular biblical scholarship fascinating and helpful).
Secular humanism has been accepted as a religion for First amendment purposes.
Non theistic Buddhism is still considered a religion.
Worldview might be a better term for us to use in understanding people's first amendment rights. And also the idea that there is no neutral worldview. E.g. "secularism" is directly hostile to many religious viewpoints, and to the degree that its proponents use it to oppose religion, secularism becomes a religion/worldview/belief system.
