Describing data, taught from zero

1 · Why engineers need statistics at all §1.1

Pick up a resistor marked 1 kΩ and measure it precisely: 987 Ω. Another from the same reel: 1,012 Ω. A third: 996 Ω. Nothing is broken — variability is a fact of every manufacturing process and every measurement. Identical inputs, identical machine, different outputs. Statistics is the discipline of making honest statements in the presence of that variability, and that's why every engineering degree teaches it.

Two words carry the whole course, so let's get them exactly right:

A population is the complete collection of objects you actually care about — every resistor the factory will make this year. A sample is the subset you actually measured — the 50 you pulled off the line this morning. Nearly always, the population is what you want to know about and the sample is all you can afford to look at. (Measuring everything — a census — is usually impossible or absurd: some tests destroy the item.)

That one distinction splits the whole subject in two:

• Descriptive statistics (this chapter): summarising the data you have — pictures and a few well-chosen numbers.
• Inferential statistics (later in the course): the leap from sample to population — "based on these 50 resistors, what can I say about the whole year's production, and how confident am I?" Probability (Chapters 2–4) is the bridge that makes the leap honest.

So Chapter 1 is deliberately modest: no leaps, no probability — just learning to look at data properly. Don't let the modesty fool you; the vocabulary defined here (especially x̄ and s) gets used in every single later chapter.

2 · First, always: a picture §1.2

A list of 60 numbers is unreadable. The first professional reflex with any new dataset is to draw it, and §1.2 gives you three hand tools:

A stem-and-leaf display — split each value into a stem (leading digits) and leaf (last digit), and stack the leaves on their stems. Nine bond-strength readings — 62, 65, 58, 71, 64, 68, 55, 66, 73 — become:

Thirty seconds of work, and you can suddenly see the data: clustered in the 60s, no stragglers, roughly symmetric. Best for small datasets (say, under 50 values) — and unlike a histogram it loses nothing: every original value is still readable.

A dotplot — one dot per value along a number line, stacking repeats. Same job, better when values repeat or when comparing two small groups.

A histogram — the workhorse for larger datasets. Chop the measurement axis into equal-width intervals ("classes"), count how many values land in each, and draw a bar of that height over each class. The bars' shape is the data's shape.

Here's the part that's easiest to learn by touch: the picture depends on how many classes you use. Below are 60 real-ish component lifetimes. Slide the control and watch the same data tell different stories:

What you should notice: with 3–4 classes the data looks like a featureless lump (too much smoothing); with 20+ it's jittery noise (too little). Around 8–12 classes the true story appears — a peak near 150–200 hours and a long tail of survivors stretching right. A reasonable default the book suggests: number of classes ≈ √n (here √60 ≈ 8). There's no single "correct" histogram — which is itself worth knowing before you trust anyone else's.

That long right tail has a name you'll use constantly. A histogram is symmetric if the two halves mirror; positively (right-) skewed if the right tail stretches further; negatively (left-) skewed if the left one does. It's unimodal with one peak, bimodal with two (bimodal usually whispers: two different populations got mixed — two machines, two operators).

One refinement before moving on, because it's a classic mark-loser: all of this assumed equal-width classes. If the classes have unequal widths (books and instructors do this for data with long tails), bar height must be density = relative frequency ÷ class width, so that area, not height, represents proportion. Otherwise a wide class looks artificially tall just for being wide. The notes page works a full density-histogram example — worth a careful read, because "draw the histogram" on an exam quietly becomes this whenever widths differ.

3 · The centre of the data §1.3

Now we compress the picture into numbers. First: where is the data centred? Two honest answers, and the gap between them is the interesting part.

The sample mean is the familiar average. For sample values x₁, x₂, …, x_n:

(Notation matters here: x̄ — "x-bar" — is the sample mean; the population mean gets the Greek letter μ. Same recipe, different data, and the whole of inference later is about using x̄ to guess μ — so the book is strict about the symbols, and exams inherit the strictness.)

Physically, the mean is the balance point: print the dotplot on cardboard and it balances on a knife-edge at x̄. That image explains the mean's one weakness — a single far-out point has enormous leverage, like a child sitting far along a see-saw.

The sample median x̃ ignores leverage entirely: sort the data; the median is the middle value (odd n) or the average of the middle two (even n). Watch them disagree. Five commute times, in minutes:

One horror commute dragged the mean to 39 — a number that describes none of your days. The median calmly reports 26. Neither is "right": the mean answers "what's my total time over many days, per day?" (leverage is the point — every minute counts), the median answers "what's a typical day?" (leverage is a lie). Choosing the honest summary for the question asked is the actual skill §1.3 teaches, and skew is the tell: mean pulled toward the tail, median not. Income statistics quote medians for exactly this reason.

Between the extremes sits the trimmed mean: delete the smallest and largest few percent, average the rest. A 10% trimmed mean of 10 values deletes the top and bottom value and averages the middle 8 — robust like the median, but still using most of the data like the mean. (The notes page's Worked Example 1 computes all three on one dataset — read it after this section.)

Finally, generalising the median: the median splits the sorted data 50/50, and quartiles split it 25/25/25/25. Devore's version of quartiles is called fourths: the lower fourth is the median of the smaller half of the data, the upper fourth the median of the larger half. Percentiles push the same idea to any split. These become the skeleton of the boxplot in section 5.

4 · The spread of the data §1.4

Two datasets can share a centre and be utterly different: {49, 50, 51} and {0, 50, 100} both have mean 50. A centre without a spread is half a description — for an engineer often the less important half. (A bridge designed for the average load fails; it's the spread that kills.)

The crudest spread measure is the range, largest minus smallest — two data points' opinion, everyone else ignored. We can do better by asking: how far is each point from the mean, typically? The deviations x_i − x̄ capture this, but averaging them raw gives exactly 0 every time — negatives cancel positives (the balance point again!). So square each deviation first, making everything positive and punishing big deviations hardest. That gives the sample variance:

and its square root, the sample standard deviation s, which undoes the squaring so the answer is back in the original units (hours, ohms, minutes — not hours²). Small worked pass, on {2, 4, 6, 8}: mean x̄ = 5; deviations −3, −1, 1, 3; squares 9, 1, 1, 9, summing to 20; so s² = 20/3 ≈ 6.67 and s ≈ 2.58. Read s as "a typical distance from the mean" and you'll rarely be misled.

Now the question everyone asks — why divide by n − 1 and not n? Two ways to see it, both worth having:

• The deviations aren't all free. They must sum to zero, so once you know n − 1 of them, the last is forced. Only n − 1 independent pieces of information — "degrees of freedom" — went into the sum, so that's what we divide by.
• The sample mean flatters the sample. x̄ is computed from these very points, so it snuggles closer to them than the true population mean μ does — making the squared deviations systematically a bit too small. Dividing by the smaller number n − 1 compensates by just the right amount, on average.

School formulas divide by n — that's the population variance σ², correct when you truly have every member. With a sample (almost always), it's n − 1. This single discrepancy with 11th-class habit is the most reliable mark-loser in the chapter.

5 · The boxplot: five numbers, one picture §1.4

The last tool marries pictures and numbers. Take the median, the two fourths (section 3), and the extremes — five numbers — and draw: a box from lower fourth to upper fourth, a line at the median, whiskers out to the most extreme non-suspicious points. The box holds the middle 50% of the data; its width is the fourth spread f_s = upper fourth − lower fourth, a spread measure that (unlike s) shrugs off outliers.

And the boxplot comes with a built-in outlier detector — the first objective rule you've met for a word people usually use by vibes:

Quick pass on the sample 4, 5, 6, 7, 7, 8, 9, 10: median (7+7)/2 = 7; lower half {4, 5, 6, 7} has median 5.5 (lower fourth), upper half {7, 8, 9, 10} has median 8.5 (upper fourth); so f_s = 3 and the outlier fences sit at 5.5 − 4.5 = 1 and 8.5 + 4.5 = 13. Every value is inside — no outliers, whiskers run to 4 and 10.

Why you'll actually use this: side-by-side boxplots are the fastest honest comparison of two processes ("supplier A vs. supplier B") — centres, spreads, skew and outliers all visible in one glance. The notes page's Worked Example 3 runs the full exam version, outlier verdicts and all.

6 · You're ready — what to do next

That's all of Chapter 1: population vs. sample, three pictures, two-and-a-half centres, spread by s and by f_s, and the boxplot. To turn it into marks:

• Work the examples: the notes page has three fully computed exam-level examples (location trio, density histogram, boxplot with outliers) plus the practice table for Devore's exercises — odd numbers, answers in the back.
• Revise from the notes page, not this one. This page is deliberately slow; before a quiz you want the compressed map.
• In the lab: the notes page's §1.5 "The same in R" maps each R function on the Week 3 sheet to the idea on this page. It also corrects the sheet's summary table: its mean, SD, variance, skewness and kurtosis are wrong. If your R output disagrees with the sheet, R is right.