I generally find that the "I'm afraid we'll have to agree to disagree" comment works for me.
No matter how dumb the original comment was.
If the original comment was dumb and offensive,
the second option of never speaking to that person again is valid.
If the person making the comment is family, be civil to them, but make sure that they are completely written out of your will.
I hope the above helps ;-)

MORGA:

I'm afraid we'll have to agree to disagree.

I hear you. I use that line a lot myself. I think it's the most effective way of saying "I don't agree with what you're saying, but I really don't want to have a fight about it either."