It depends. If you are calling someone crap then I think it would be a swear word. But if you are just saying it like oh crap I forgot my books i think it’s fine.
I personally find it a swear word if it's directed at someone. If you're saying something like: "Oh crap I missed class" I'm fine with it, but that's just my opinion.