MGFs, Chebyshev and transformations, taught from zero

1 · A barcode for curves L12–16

In Module 2's lesson the moment generating function was a barcode: one function, M(t) = E(e^tX), that carries every moment of X and identifies its distribution. Differentiate and set t = 0: out comes the mean; once more, E(X²).

For a continuous random variable, nothing about the idea changes. Only the arithmetic does, the same way it did all through Module 3: the sum over values becomes an integral over the density.

Take the friendliest density, the exponential, f(x) = λe^−λx for x ≥ 0. Put it in, and the two exponentials merge into one, since e^tx · e^−λx = e^−(λ−t)x:

Here is the one genuinely new wrinkle. That integral runs to infinity, and it only has a finite value if the integrand dies away — if λ − t is positive. Try t = 2λ: the integrand becomes e^+λx, which grows forever, and the area is infinite. So the barcode only exists for t < λ. That's no problem — we only ever use t near 0 — but the range is part of the answer, and you should write it down every time.

For t < λ, the integral of e^−cx from 0 to ∞ is 1/c, so

Check the barcode's one universal feature: M(0) = λ/λ = 1 ✓ (it's the total probability). Differentiate: M′(t) = λ/(λ − t)², so M′(0) = 1/λ — the exponential's mean, as promised. The notes page finishes the variance and does the gamma the same way; its table lists all five MGFs you should know by heart for a closed-book paper.

2 · The normal's barcode, and reading it backwards

The normal's MGF is the one exams love, and its derivation hinges on one piece of algebra worth doing slowly. For the standard normal Z, the integral is

The exponent, tz − z²/2, is almost the exponent of a bell curve, −(z − something)²/2. Completing the square makes "almost" exact. Start from the square we want and expand it: (z − t)² = z² − 2tz + t². Halve it and put a minus sign in front: −(z − t)²/2 = −z²/2 + tz − t²/2. Compare with our exponent: it's the same except for the extra −t²/2. So add that back:

Now e^t²/2 doesn't involve z, so it walks out of the integral, and what stays inside is the density of a normal centred at t — the standard bell, slid sideways. A density's total area is 1. So M_Z(t) = e^t²/2. Stretching by σ and shifting by μ (X = μ + σZ) turns it into

(the notes show the stretch-and-shift rule in one line).

Now the move that actually earns marks: reading it backwards. Because an MGF identifies its distribution uniquely, an exam can hand you an MGF instead of naming the distribution. Your job is to spot the shape and read the parameters off it. For a normal: the coefficient of t is the mean; the coefficient of t² is half the variance.

The 2025 mid-sem opened exactly like this, for 9 marks: exam-completion time has MGF e^{0.5(σ²t² + 160t)}. The 0.5 in front is disguising it. Multiply through: 80t + σ²t²/2. Normal, with mean 80. The rest of the question (finding σ from a given tail probability, then two table lookups) is pure §4.3 — the notes' Worked example 3 does it in full, with a figure.

3 · When barcodes converge

Uniqueness has a second, more surprising use. Suppose you have a whole family of random variables — one for each value of some parameter — and you want to know what the distribution looks like as the parameter goes to an extreme. Computing the pmfs and taking a limit is usually hopeless. But if the MGFs settle down to the MGF of a known distribution, the distributions settle down to it too. So you take the limit of one formula instead of a whole distribution.

The 2025 mid-sem's final question (12 marks) was exactly this. X counts the trials needed for the r-th success; its MGF is [pe^t/(1 − qe^t)]^r. When p is tiny, X is huge (you wait ages for each success), so the question rescales it: Y = 2pX. Watch what happens to Y's MGF as you drag p toward 0:

The solid curve climbs onto the dashed one. The dashed curve is (1 − 2t)^−r — from section 2's second check, that's a gamma with α = r, β = 2, which is chi-squared with 2r degrees of freedom. So for small p, 2pX is approximately χ²(2r).

On paper, the limit needs one fact from school calculus:

It's the slope of e^x at x = 0 written as a limit (rise e^h − e⁰ over run h), and that slope is e⁰ = 1. The trouble in Y's MGF is that numerator and denominator both go to 0 as p → 0. Dividing both by p turns the denominator into a piece of exactly that shape, and it tends to 1 − 2t. The notes' Worked example 4 does it line by line, including part (a)'s MGF derivation.

4 · Chebyshev: a guarantee from two numbers L19

Everything so far assumed you know the distribution. Suppose you don't. A supplier tells you only that their bolts have mean length 50 mm and SD 2 mm. Can you say anything about how many bolts lie far from 50?

Surprisingly, yes. Chebyshev's inequality says that for any distribution at all, and any k > 0:

Read it as: at most 1/k² of the probability can be k or more standard deviations from the mean. At most a quarter lies 2 SDs out; at most a ninth lies 3 SDs out. Flip it and you get a floor: at least 1 − 1/k² lies strictly within k SDs. For the bolts, at least 1 − 1/4 = 75% are within 46–54 mm, whatever the distribution is. (The handout asks for the statement without proof.)

The price of knowing so little is that the guarantee is usually loose. Slide k and switch shapes to see how far the true tail sits below the bound:

Three things to notice. The shaded area is always below the bound — that's the theorem. For k ≤ 1 the bound is 1 or more, so it says nothing at all. And the exponential, which is lopsided, has no left tail beyond 1 SD (it can't go below 0), yet Chebyshev still holds for it — it holds for everything.

Exam questions run it forwards ("at least what fraction lies between 440 and 560?") or backwards ("how wide must the interval be to hold 96%?" — solve 1 − 1/k² = 0.96, giving k = 5). The notes' Worked example 5 does both, plus the trap of an interval that isn't centred on μ.

5 · Transformations: new distributions from old L20

Last idea. You know how X is distributed; someone computes a new quantity from it — Y = X², or Y = −ln X, or a temperature converted from °C to °F. How is Y distributed?

The tempting move — plug g into the density — is wrong, because a density isn't a probability. The safe route goes through something that is a probability, the cdf. That's the whole cdf method:

1. Write F_Y(y) = P(Y ≤ y) and replace Y by its formula in X.
2. Solve that inequality for X. (Careful: multiplying by a negative number flips it.)
3. You now have a probability about X — compute it. Differentiate if you want the density.

Try it on a famous one. U is uniform on (0, 1) — a calculator's random-number button — and Y = −ln U. Since U < 1, ln U is negative, so Y is positive. Then Y ≤ y means −ln U ≤ y, which means ln U ≥ −y (the minus flipped it), which means U ≥ e^−y. For a uniform on (0, 1), the probability of landing in an interval is its length, so P(U ≥ e^−y) = 1 − e^−y. That's the exponential cdf with λ = 1. The random-number button, passed through −ln, produces exponential waiting times — this is how simulations generate them.

One shape needs extra care: a g that goes down and then up, like z². Then "Y ≤ y" catches values of Z on both sides of 0:

For Z ~ N(0, 1) that gives F_Y(y) = 2Φ(√y) − 1, and differentiating turns out to give the chi-squared density with 1 degree of freedom: the square of a standard normal is χ²(1). That's the chi-squared's origin story, and why it keeps appearing in the statistics half of the course. (Full derivation: notes, Worked example 6.)

When g only ever increases (or only decreases), the cdf method always produces the same shape of answer, which the notes package as a formula: f_Y(y) = f_X(x) · |dx/dy|, with x written in terms of y. The |dx/dy| factor corrects for stretching: if g spreads an interval out, the same probability covers more width, so the density there must be lower.

6 · You're ready — what to do next

The part in one sentence: an MGF is a barcode you can derive (an integral), read backwards (uniqueness) and take limits of; Chebyshev bounds tails using only μ and σ; and the cdf method turns "distribution of g(X)" into a probability about X. Next:

• Work the examples: the notes page has seven worked examples (two are real 2025 mid-sem questions), the MGF catalogue, and the traps list.
• Then the past papers: the mid-sem PYQ page and the mid-sem drill.
• Earlier material: Module 3, Part 1 (densities, normal, exponential) is the base all of this stands on.