How does a native speaker ( be english, spanish of russian whatever) know that an other person is not native?

Oh and please don't say because of accent, bullshit, I can imitate a Scottish accent pretty well but so it not important.lol. So how? becaouse of vowel sound or what. ps Charlize theron's US accent pretty good by the way.(damn she's so hot XD) What a question jeez and i'm not even drunk lol