Did you know that 50% of Americans are of less than average intelligence whereas 50% of the British are over average intelligence?
If this statement confirms your beliefs, makes you annoyed or in fact arouses any emotion other than amusement, this article is meant for you.
Statistics is the study of evaluating data. And today, more than at any time in history, we are flooded with data. Data on the usefulness of seat-belts, different things to do to reduce pollution, effect of smoking on lung-cancer, etc. etc. With home computers and data-banks it is getting worse and worse. What should we do with all these facts and percentages? Just memorize the more spectacular ones and use them to win arguments? Or should we try to be a bit more critical? If you think "yes", read on.
Take an example. "A recent survey of smoking in Europe revealed that 84% of French men smoke more than 10 cigarettes a day".
There you are. A nice shiny fact. Compare it to the fact that 34% of Germans smoke (revealed last week), and you can see that the French smoke about 2.5 times as much as the Germans. Which just confirms what you knew already anyway. Remember when you were in that bistro in Paris and those two men were smoking those filthy Gaulois cigarettes? Whereas in Germany ... why in Munich they even had a non-smoking area in McDonalds.
Well, remember we are going to be a bit critical about these figures. Let's look at the `fact' that 84% of the French smoke. Where did this fact come from? Did someone really knock on doors all over France and ask sixty million French people "Excuse me Monsieur" (or Madame or Mademoiselle) "Do you smoke more than 10 cigarettes per day?" A pretty mind-boggling task you would think, cost a bomb, who paid for it and was it worth it. But let's assume it really was done. Just think, 84% of the French smoke more than 10 cigarettes a day, that means 25.2 million packets a day, or put end to end 8316 times the height of the Eiffel Tower (per day)!
But don't get excited. Let's think more about the validity of this 84%. Suppose someone asked you how many cigarettes per day you smoked (assume you do smoke, for the sake of argument).
"Well, sometimes I smoke 5, but if there's a lot of stress at the office it can go up to 20." Actually you buy a carton of 200 per week and smoke most yourself which works out as ... but you shy away from making this calculation.
Same with the French. Some put down less because they have promised the wife to cut down, others put down more because it's considered macho, yet others put down anything, because they're French. That nice clean sharp 84% is already starting to look a little frayed at the edges.
To confuse the issue further, cigarettes are pretty expensive in France - the price is always rising - so it would be nice to know some background. Has the French government recently raised the tax on cigarettes? Or has there recently been a `non-smoking' campaign?
It's beginning to look as though that precise figure of 84% really just means that "a lot of French smoke cigarettes" which you knew already.
But we'll trace the figure to its source and call up the agency that supplied it.
Me "Is that the DL Agency? I'm interested in that figure you gave to Time Magazine the other day. quoting 84% of French smoking more than 10 cigarettes per day."
DL "84%?" Rustle of papers. "Yep, that's it. Came in last week. I've got it on the computer here."
Me "Excuse my curiosity, but 84% seems a very accurate figure. May I ask how you arrived at it?"
DL "Well, I don't do those projects myself. I think ..yes, we sent Joe to do it."
Me "Joe did it. That must have been some big job. There are sixty million French and say 1000 a day. that works out at ..164 years, so .."
DL (aside) "No, it's some nut. He thinks we counted every goddam Frog. (louder, to me) "Ah, no sir. I'm sure Joe only checked a sample of Frenchmen."
Me (curiously) "A sample? How big a sample? It would have to be a sample of typical Frenchmen."
DL "Sure, sure. Why don't you let me put you through to Joe." Click, buzz.
Me "Is that Joe? I understand you recently did a survey of European smoking habits and came up with a figure of 84% for the French. Could you explain where you got that figure?"
Joe "Sure. My boss sent me down to Kennedy airport and I got hold of the receipts of the Duty Free shop and I checked how many people on the different flights to Europe bought cigarettes."
Me "You mean you checked all Air France passengers who went through Duty Free that day and bought cigarettes, to find the proportion of French who smoke?"
Joe "Oh, no sir. That would be very inaccurate. I only counted those who gave their addresses in France. The DL Agency has a reputation for accuracy to maintain".
Me "To come back to the French. How many passengers did you actually count that day?"
Joe "Just a minute, sir. I have the data right here." (clicking of computer keyboard) "2350 passengers with French residence going to Paris, of whom 1974 bought cigarettes which makes.. " (more clicking) "84.00%"
Me "I see. But what about the `more than 10 cigarettes smoked a day' which you had in your survey?"
Joe "Oh, that. We got that from national statistics. It's well known that anyone who smokes at all usually smokes at least 10 per day."
Me "Thank you Joe. You've been very informative."
Joe "My pleasure."
Mein Gott. You put down the telephone feeling the statistic of 84% is not worth the powder to blow it to hell, and you would be right. Further errors you can see immediately:
- Not everyone who buys cigarettes in a Duty Free shop is a smoker: they are often being bought for friends.
- You don't have to be French to live in France. How many of the passengers were actually French?
- And assuming they were all French and all smokers, how typical of the general French population were they? Only a very small proportion of the French fly to the US.
And we haven't got to the “statistical errors” yet!
You might think that the above case history is exaggerated, but it is very typical. It uses new data, and is by present-day standards painstakingly accurate. Most editors, wanting to quote the number of cigarette smokers in France, would look up some data collected amongst Breton fishermen by the Marlboro Company in 1963.
Now, as mentioned before, we live in a world flooded with data: "57% of the population is blue-eyed", "Reducing speed on the motor-ways to 100kmph will reduce exhaust emissions by 14% and save 18% of the woodland", "The Conservative Party is 11% ahead of all other political parties", "Fluorine in drinking water saves 83% of teeth from caries"...
These figures pretend to define something which is `real'. There is a fixed proportion of the population which has blue eyes, one of the political parties is ahead of the others, there is an average age at which white male Caucasians die. But how to find these figures? That is the problem. Take eye colour, for example. It's just not possible to observe everyone in the Kingdom and write down whether they have brown or blue eyes. It could be done and it would give the `right' answer, but it would take a lot of effort. The science of Statistics was developed to find an easier way.
Statistics doesn't attempt to collect all the data, that would be too expensive and time consuming, statistics "samples" the data. A statistician just looks at (say) a thousand of the population and finds out how many have blue eyes. From this he makes an intelligent guess as to the proportion of blue eyes in the whole population.
Fairly easy to do, you would think. After all, you just have to look at them. They can't hide the fact that they have brown or blue eyes. They can't lie about it, like they can about cigarette consumption. But think a bit more. Where do we get this sample of 1000? It's no good looking in Scotland, where they almost all have blue eyes. Nor would it be much use looking in inner-city Birmingham, which has a large number of black immigrants. Ring up 1000 people and ask them their eye colour? Brown-eyed immigrants don't have telephones.
So the big problem in Statistics is to find a Representative Sample. And it doesn't get any easier if you have to take a smaller sample - the smaller the sample the more likely you are to make a mistake and take a `biased' sample.
But let's say you have finally done it. Your sample has been taken from all over the country - 1000 people from all walks of life, living in all geographical locations, mixed up English, Welsh, Scots, Irish, immigrants. You have personally checked their eye colours and you have a figure. How good is it?
Now you have taken a sample of 1000 out of sixty million Britons. Do you have a sneaking feeling that it was all a bit too easy? Well, your suspicion is well founded. You can't fool around with Nature: you only sampled 1000 so you have to `pay' for it somewhere. And the payment is that your answer is not very precise. You cannot have too much confidence in it. It is the business of the science of Statistics to show you how good your answer is.
Now before we move on to the Statistics part of this essay, let's consider what is meant by Error, because Statistics has a rather unusual way of talking about errors.
If you add 200 and 300 and come up with 498 you have an error. Not a bad one, only a 0.1% error, but it would fail you in a maths exam. In maths no errors are allowed. But in real life you're damn lucky if you can do anything to 0.1% accuracy. Estimate your gas consumption from Birmingham to Liverpool to 0.1%? Keep your speed constant to 0.1%? Work out your weekly grocery bill to 0.1%? Very difficult.
But the answer to a problem in Statistics comes out in a much more realistic or user-friendly form. Answers in Statistics look like: 42 (CL +4), 568 (CL+9,-7), 24%(CL 0+2%). CL means Confidence Level.
Translation - 42 (CL+4) means "I'm pretty sure the real answer lies between 38 and 46. It could be outside these limits (ie. 36 or 50), but it's unlikely. As in: my wife tells me she will meet me at a shopping centre at 2 o'clock. By experience I know this means 2 o'clock (+5mins,+20 mins). I'm confident she will be there between 2:05 and 2:20. I would be surprised if she appeared outside these limits. These limits are called Confidence Limits and are similar to the tolerances you have when you buy steel girders or electronic components.
What Statistics does is to enable you to estimate what these limits are under certain conditions and tell you what you must do if you want them to be tighter. You might guess that in the `blue eye' problem above, the way you `pay' for taking a small sample of the population instead of the whole population is in having wide confidence limits ie. not being very sure of the answer.
The +2ÖN rule
Here is an interesting example attributed to Lord Kelvin (the one who said your knowledge of anything is vague and unsatisfactory unless you can measure it). Suppose you mark all the molecules in a glass of water and then pour the glass into the ocean and allow them to mix throughout the seven seas. If you then took the same glass and dipped it into any one of the seven seas you would find about 100 of the marked molecules in your glass.
There will be a short pause while you gasp over this. Yes, there are a lot of water molecules in a glass of water and that was what Lord Kelvin was trying to illustrate.
But right now we are interested in the fact that there are `about' 100 of the original marked molecules in the glass of water you have fished out of one of the seas. What this means is that if you kept on dipping your glass into the oceans and each time counting the number of marked molecules in it, you would find the numbers to be something like: 104,97,92,101,109,100, 110, 92 etc. In fact, let's imagine someone actually does just that, about three or four thousand times and makes a list. Then he goes through his list and counts up how many were the same. Part of his list might look like:
No. of marked molecules No. of times
found in the glass
He would then plot these on a piece of graph paper and the result would look very much like that below. You can see where he plotted the fact that when he looked in the glass he found that 300 times there were exactly 114 marked molecules.
75 80 90 100 110 114 128 130
No. of molecules in a glass
This is a very important graph. It is called a Gaussian Curve after Herr Gauss, the man who discovered it. What can we say about this curve?
- Its maximum value shows 1000 glasses contained 100 marked molecules.
- There were no glasses that had less than 75 or more than 130 marked molecules in them.
- In fact, because of the way the Gaussian curve flattens out at the extremities, you can say that "almost all" (actually 95%) of the glasses had between 80 and 120 marked molecules.
This means (and don't skip this bit!) that if he dips his glass once more into the ocean and counts the number of marked molecules:
- He will almost certainly find between 80 and 120
- The most likely number is 100.
Now the numbers I have used here are exact, this is what would actually happen. If the glass he used was 10 times bigger, he would (as expected) fish out 10 times more marked molecules ie. average value 1000 per glass instead of 100, but this time he would find that the variation from glass to glass was less. The variation (or dispersion) between most and least would only be 1000+63 (instead of 100+20).
In order to calculate these figures, I am using a well know formula:
Dispersion = + 2ÖN
where N = average number of marked molecules
The +2ÖN rule enables us to say something about the accuracy of a measurement. It gives you the Confidence Level. In the Smoking Frenchmen case above, if we ignore all the sampling errors, everything was decided on finding 1974 people bought cigarettes. The dispersion of this figure is +2ÖN or 1974+88 so although the percentage quoted was 84%, it could in fact be anywhere between:
1886/2350 = 80%
and 2062/2350 = 88%
So the next time you hear a statistic ask yourself:
- What is the sample size? Apply the +2ÖN rule and you will get the Confidence Level and be able to see how valid the conclusions are.
- Was a representative sample taken? Be critical. What sort of errors could creep in and how were they avoided? Try to imagine how the data was actually taken and you will usually see plenty of possibilities of mistakes being made.
Above average intelligence
Now the reason for the outrageous claim at the beginning of the "Statistics" may be clearer. If you measure the intelligence of all Americans (however you do that, whatever intelligence means… ) you will find that they will all be different, a few genius level, a few morons, but most will be of the same or of "average" intelligence. In fact they will be distributed as in the Hr. Gauss’s curve. This Gaussian curve is symmetrical which means that 50% of Americans are above average intelligence and 50% below. There are also 50% of British above average intelligence and 50% below. You start the argument by quoting half of one statement and half of the other!