So it would have to get to the point where I know a dozen people (including myself) who've been victims of right wing violence?

The overall robbery rate was 119 per 100k (http://www.fbi.gov/about-us/cjis/ucr/crime-in-the-u.s/2010/crime-in-the-u.s.-2010/violent-crime/robberymain) in 2010. Think "right wing violence" that's driven by violent political rhetoric even comes close to what would be considered muggings in that data?

If right wing violence becomes that frequent, what do you think is likely to happen to political freedom in this country?

It won't, don't worry.