No, "Caramba!" is not a swear word.
The word "Caramba' is very common exclamation in the spanish-speaking countries and does not have an specific meaning, being used to express surprise either positive or negative.
It is not vulgar or offensive and its English equivalent could be "Gee!" or "oh my God!".