There are lots of different accents in different parts of Britian, the US and Canada, but it's true that in a very general way you can usually tell, if you listen to an English speaker, whether he or she comes from North America or Britain (of course there is also Australia, New Zealand etc which are different again.)

Many of the differences can be traced to the first English speaking colonists who arrived in America in the 16th and 17th century. English then would have sounded much more similar to American today; especially, the rhotic 'r' was in use (when 'r' is strongly pronounced even inside or at the ends of words, eg in garden or liver. This is typical of English West Country and Scottish accents even today. The longer American 'a' can also be traced to 17th century English origins.

You can learn more about the differences between these accents here.