When I was an undergrad, a friend showed me a version of the following famous paradox. Suppose somebody puts money into two envelopes, with one envelope getting thrice as much money as the other. You pick an envelope at random and see that it has $90. Now you have the option to exchange the $90 for the money in the other envelope. Should you do it? Well, not having much information, you figure it's about as likely that the other one has $30 as it does $270 and compute that in expectation you'll get 0.5($30)+0.5($270) = $150 if you swap. This of course works for any amount x -- you can get about 1.67x in expectation by swapping. You choose the envelope at random but always want to swap for the money in the other envelope -- this is the paradox! Does it trouble you?
It shouldn't -- because I cheated here. I never said how the money was placed in the envelopes. For example, because there are infinitely many choices for possible amounts of money you can place in the envelopes, you can't choose from them uniformly at random. And there are lots of intuitive distributions that won't work; perhaps the whole setting is impossible.
So let's be more careful. Let's say the person placing the money chooses a positive integer y with probability 2^(-y) (this is an honest distribution) and then places 3^y dollars in one envelope and 3^(y+1) dollars in the other. Now, when you open an envelope at random and see x dollars, you can figure out the probabilities of there being x/3 and 3x in the other. If x = 3, you know you'll win by switching because the other envelope must have $9. Otherwise it's not hard to see that you'll get x/3 with probability 2/3 and 3x with probability 1/3, giving you about 1.22x in expectation for switching -- not exactly the 1.67x, but still a gain.
Now again we get the same paradox: you choose an envelope at random but always win (in expectation) if you switch -- this even works if you don't look at the amount in the envelope! Does this break probability or only intuition?