I have a theory that perhaps north Americans have more difficulty understanding other English speaking accents than vice versa because North American TV and movies are such a big part of overseas people's lives, but the opposite not so much true. British TV is something I've grown up with in Australia, so I'm kind of used to Scottish, northern English accents, etc . But we lived in Canada for a year, and when some UK people came over to visit, a lot of Canadians (native English speakers) had difficulty understanding them, but we didn't.
I proudly announced this theory to my partner, and they told me I was an idiot :)