A Devious Bet 2: Law of Large Numbers

You meet up with a slightly familiar mustached squid “man”

He may or may not have bankrupted you infinitely times over in the past, but you’re over it. In fact he has a new deal for you!

Every day he will modify your bank balance in one of two ways. With probability 0.6, your balance will get multiplied by 1.7, and with probability 0.4, it will get multiplied by 0.45 (independently from the other days or how much money you already have)

“Now then” the squidish man continues. “Ignore the oddly specific constants and look at the facts”

“Each day, on average, your balance multiplies by 1.2. If you started with 1 dollar, tomorrow you will have 1.2 dollars on average, in a week about 3.58 dollars on average, and in 100 days, 82817974.52 dollars on average. You are expected to become a millionaire, should you take my deal”

You think about it for a bit. The deal seems fair, on average you earn each day, no? Plus the inky man has a cool mustache, he can’t be a liar with faical hair like that!

Nevertheless, you agree and take the deal for a year. 5 years later, you look at your bank balance.

Oh, its less than a dollar. Well at least you don’t have infinite debt like last time…

Why it happened

Suppose you started with only one dollar. We denote Sn your total money on day n, so S01. Moving from the (n-1)th day to the nth, your amount of money gets multiplied either by 1.7 or 0.45. For ease, let’s call that (random) amount Xn. With probability 0.6, Xn is 1.7, and with probability 0.4 it’s 0.45. Additionally, as stated before, those X-s are independent from each other, so those probabilities aren’t affected by what the other X-s were

So for example, S1 will just be X1, S2 will be X1X2. In general, the amount of money you have on day n (Sn) will be the product of the first n X-s:

$\displaystyle{S_n=X_1\cdot X_2 \cdots X_n =\prod_{k=1}^{n}X_k}$

From the calculation in the previous section, the expected value of Xn is 1.2, seemingly in your favor as it’s greater than one. Using independence of the X-s, the expected value of Sn is (1.2)n. This expected amount of money grows exponentially large as n grows. The “totally not a squid”-man wasn’t lying when he said that, on average, you will be a millionaire after a 100 days

Here is the thing though, a high expected value does not guarantee a high probability of winning. For example, think of an event where, with probability 0.999, you lose 100 dollars, but with probability 0.001 you win 1,000,000 dollars. Your expected revenue is a relatively high 0.001*1,000,000-0.999*100=900.1 dollars. But more often than not, you will just lose a 100 dollars. The high expected value here isn’t really in your favor to win, it’s in your favor when you do win.

How exactly do we show that your probability of winning is very low? For that we have to use a theorem in probability known as the law of large numbers:

$\textbf{Theorem:} \textup{ for independent identically distributed random variables }X_n, \textup{such that:}$
$\mathbb{E}(X_n)=m, \mathbb{E}(X_n)<\infty \textup{ for all n}$
$\textup{Then almost surely: }\displaystyle{\lim_{n\rightarrow\infty}\frac{1}{n}\sum_{k=1}^nX_k=m}$

Let’s digest what all this means. We have an infinite array of random variables, each independant from each other, and each distributed the same way. For example, we can have infinite coin tosses of fair coins with sides 0 and 1. The expected value of each such variable is known as m, in the coin example m=0.5. The restriction of the expected value of the variable squared is more of a technicality, so we don’t get “”weird”” variables. In most cases this restriction holds so we don’t really need to worry about it.

What the theorem says then, is that almost surely, that is with probability one, the average of the first n values we got will approach the expected value m, as n goes to infinity. The more values we get, the more their average will approach the expected value of a single variable. Going back to the coin toss, if you say toss a 1000 such coins, with a very high probability the average value of the tosses will be close to 0.5

In actuality the law of large numbers doesn’t guarantee a high probability specifically for n=1000. To show specifically for that value of n, you need other tools such as the central limit theorem or Chebyshev’s inequality (possible subjects for an upcoming post)

This all sounds great and all, but what does it all have to do with the bet from above?? In the bet there was no average of the variables, and not even a sum of them but a product. How those the law of large numbers apply here at all?

Well, the trick is using logarithms. Instead of looking at the amount of money you have on each day, we look instead on the log of that amount (for this example the log will be base e, but the base doesn’t really matter here). Why do logarithms help us here? Remember the the log of a product, is the sum of the logs of each term in the product. Applying that for our example, we get the following:

$\log(S_n)=\log(X_1\cdot X_2 \cdots X_n)=\log(X_1)+\cdots +\log(X_n)$

We now have a sum of new variables, still identically distributed and independent, log(Xk). Let’s use the law of large numbers then. The expected value of log(Xk) will be:

$\mathbb{E}[\log(X_k)]=0.6\cdot\log(1.7)+0.4\cdot\log(0.45)\approx -10^{-3}$

Which, to our surprise, is negative! So as n grows, the average of the first n log(Xk)-s will approach that expected value:

$\frac{1}{n}\log(S_n)=\frac{1}{n}[\log(X_1)+\cdots+\log(X_n)] \longrightarrow ~10^{-3}$

In other words, log(Sn) will be approximately m*n for large values of n. As before, Sn is your amount of money on day n, and m≈-0.001. Taking exponent, we get that for large values of n that Sn≈e-0.001*n with high probability. So say for n=365, with high probability your amount of money on day 365 (after a year) will be around 0.69 dollars, not exactly very high now is it?

Again, the approximations done here are not exact, to prove specifically for n=365 for example, you’d need a bit more tools than the law of large numbers

Some takeaways

What can we learn from this example?

  • Probability behaves nicely with a large sample size: The law of large numbers told us that is certain cases, the average of the first n variables gets closer and closer to a constant value (the expected value of one variable) as n gets large. This more the sample size grew, the less the “randomness” of the individual variables came into play. This applies not only when taking averages, for example if we divide the sum instead by √n, the distribution will approach a specific distribution called normal distribution. As long as the individual variables hold a few restriction, that convergence will happen, in what is known as the central limit theorem.
  • High expected value is sometimes meaningless: In the example we saw, the variables Sn had a large expected value, which grew exponentially as n grew. However, with a very large probability Sn were close to 0. The reason they did have a large expected value is because that with a very, very small probability, Sn is very very large. When calculating expected value this came into account, and the small probability large value option “won”, giving us a large expected value. This applies in many cases, so not always a high average is in your favor
  • Logs are useful: We turned the product into a sum by using logarithms on it. This technique is not only useful in probability, but in many other fields of math as well. In general, if you wanted a sum but got a product, logarithms could be very useful
  • Never trusts squidish looking people: Self explanatory, you got scammed by the same “”man”” twice. Anyways, it smells a bit inky here doesn’t it?

3 thoughts on “A Devious Bet 2: Law of Large Numbers”

  1. Actually, the specific parameters chosen make this into a very good bet (just not as good as it seems by looking at the expectation). The E(log(X)) is so small that it can be considered negligible when repeating only 365 times. On the other hand, Var(log(X)) is not so small, so there’s actually:
    49% chance of breaking even,
    32% chance to get at least 10$,
    19% to get 100$,
    10% to get 1000$,
    4% to get 10000$,
    1.5% to get 100000$
    and 0.5% to get 1000000$!
    I would take that bet in an instant!
    (not including the time to calculate)

    Reply
  2. Actually you can profit reliably from this deal. If you have a pile of cash under the bed, and move money into and out of your account sometimes.

    Reply

Leave a Reply to DonaldCancel reply

%d bloggers like this: