This is like the restaurant argument:

Imagine going into a restaurant, ordering a meal, and then when presented with a bill going "What?? I dont recall signing an agreement..."

Some agreements are implicit (or explicit in this case, since the Constitution gives the government the right to raise taxes), which is akin to paying cover to be in a club...