ASTR 3130, Majewski [FALL 2012]. Lecture Notes

## ERROR ANALYSIS, PART I: SCIENTIFIC NOTATION AND SIGNIFICANT DIGITS

• No measurement is infinitely precise.

• For a variety of reasons that we will explore a little bit later, each measurement will have certain levels of associated "random and systematic errors".

• This means that when we cite a value for a measurement, it cannot be stated with exactness, but is acknowledged to have an associated level of uncertainty.

• Science has developed certain conventions for conveying a rough sense of the the level of precision in a cited numerical value via the number of cited digits in the number.

• One of the most common mistakes made by students of science is to violate these conventions and thereby convey an improper level of implied precision in a cited number.

• As a common example, students will often use their calculator to compute a number, and then record every digit put out by that calculator, even when some of those digits are spurious and well beyond where they have actual meaning compared to the true level of precision in the number.

• Mastering the conventions of stating measurements/values with the correct number of significant digits is important, and the subject of this webpage.

WEB REFERENCES:

www.chem.sc.edu/faculty/morgan/resources/sigfigs/index.html.

http://hrsbstaff.ednet.ns.ca/benoitn/chem11/units/measurement/sig_fig.htm.

### Scientific Notation

Before proceeding with the conventions of siginficant digits, it is important to review the conventions used for writing very large or very small numbers using scientific notation. In scientific notation we write all numbers in the form:

a x 10b

where:

• the exponent, b, of the base (in this case 10) is an integer,

• the mantissa (also called the significand), a, is a real number between 1 and 10.

(Note that if b = 0, we do not usually write " x 100 ", which is just unity anyway.)

So we have these examples (shamelessly lifted from Wikipedia):

Standard decimal notation Normalized scientific notation
2 2x100, or just 2
300 3x102
4,321.768 4.321768x103
-53,000 -5.3x104
6,720,000,000 6.72x109
0.2 2x10-1
0.000 000 007 51 7.51x10-9

Note that the base and exponent are often expressed in other ways. So, for example, the number 1.23 x 106 can be written in these alternative forms:

• 1.23e+6. This is called "e notation", often used in, e.g., computer programs because superscripts cannot always be displayed conveniently. Here the "e" stands for exponent; however, this format can be confusing with the "e" potentially confused with the mathematical constant e. This potential confusion can be avoided by using...

• 1.23E+6, the "E notation".

• 1.23 x 10^6 .

### Significant Digits (Also Called "Significant Figures")

• Significant Digits (Significant Figures) in a cited number are those that can be claimed to be known with some degree of reliability.

• It is presumed that as you go from left to righthand digits in a cited number, the degree of reliability in the digits decreases.

As a measurement to derive a number is made increasingly more reliably (e.g., with new experiments or measurements of higher precision, or by performing more experiments of the same precision and avaeraging the results), the number of digits/figures in the number that can be stated with confidence generally increases.

• For example, a number like 123.4 is said to have 4 significant figures, while a number like 123.45 is said to have 5 significant figures.

• An interesting historical example:

• If I say that the height of Mt. Everest is 29,029 feet, the implication is that I know the mountain is precisely 29,029 feet to "about" 1 foot accuracy, or at least better than 10 foot accuracy.

That is, I can assume with great confidence that the mountain is not likely to be 29,039 feet or 29,019 feet, but very close to (within a few feet of), if not exactly, 29,029 feet.

• But if I say that Mt. Everest is 29,000 feet high, the implication could be that I only know the height of the mountain to about 1,000 foot accuracy, because I am not giving anything but zeros for the last three digits.

• This particular example is interesting, because in 1852, an Indian mathematician and surveyor, Radhanath Sikdar, discovered that Mt. Everest was the highest peak in the world. He did this using trigonometric surveying measurements made with giant (precise) theodolites carried by teams of men led by James Nicholson (who did the hard measuring work, but came down with malaria and was forced to return home) as part of the "Great Trigonometric Survey of India" started in 1802.

Later, in 1856, Andrew Waugh, the British Surveyor General of India, using Nicholson's data, determined that the height of Mt. Everest (then called "Peak XV") was exactly 29,000 feet high.

However, Waugh was fearful that if he reported this number it would give the impression that it was nothing more than a rounded off estimate. So he arbitrarily added 2 feet to the height of Peak XV and announced publicly that the mountain was 29,002 feet high.

I think most people can appreciate that "29,002 feet" sounds more exact than "29,000 feet", but in this case, it most definitely wasn't!

• Waugh would have avoided this problem had he reported that the mountain was "29,000.0 feet high", but he didn't actually want to claim that he knew things to the level of 0.1 foot accuracy.

Following the current norms for citing significant figures (given below), if Waugh had stated that Everest was 2.9000 x 104 feet or 29,000 feet, it would be clear that his measurement was intended to be good to the foot.

As one can see, it is somewhat intuitive how people interpret numbers based on how they are reported.

The conventions now followed for stating numbers with significant digits relies on, and codifies, this intuition via a set of rules.

### Rules for the Number of Significant Figures to Quote in a Measured Quantity

These rules given here and in the next sections are adapted from those listed in the tutorial by Stephen L. Morgan at http://www.chem.sc.edu/faculty/morgan/resources/sigfigs/sigfigs3.html and at http://hrsbstaff.ednet.ns.ca/benoitn/chem11/units/measurement/sig_fig.htm.

1. All nonzero digits are significant:

• 1.234 g has 4 significant figures,

• 1.2 g has 2 significant figures.

2. Zeroes between nonzero digits are significant:

• 1002 kg has 4 significant figures,

• 3.07 mL has 3 significant figures.

3. Leading zeros to the left of the first nonzero digits are not significant; such zeroes merely indicate the position of the decimal point:

• 0.001 C has only 1 significant figure,

• 0.012 g has 2 significant figures.

4. Trailing zeroes that are also to the right of a decimal point in a number are significant:

• 0.0230 mL has 3 significant figures,

• 0.20 g has 2 significant figures.

5. When a number ends in zeroes that are not to the right of a decimal point, the zeroes are not necessarily significant:

• 190 miles may be 2 or 3 significant figures,

• 50,600 calories may be 3, 4, or 5 significant figures.

• As in the Everest example above, 29,000 feet could be 2, 3, 4, or 5 significant figures.

The potential ambiguity in this rule can be avoided by the use of "scientific," notation, because this format places the significant 0's to the right of a decimal point.

For example, depending on whether the number of significant figures is 2, 3, 4, or 5, we would write 29,000 feet as:

• 2.9 x 104 (2 significant figures),

• 2.90 x 104 (3 significant figures),

• 2.900 x 104 (4 significant figures), or

• 2.9000 x 104 (5 significant figures),

By writing a number in scientific notation, the number of significant figures is clearly indicated by the number of digits in the decimal, as shown by these examples.

An alternative method sometimes employed is using a bar over the figures intended to be significant. For example:

• 29,000 (2 significant figures),

• 29,000 (3 significant figures),

• 29,000 (4 significant figures),

• 29,000 (5 significant figures),

### Exact Numbers

Some numbers are "exact" because they are known with complete certainty and are assumed to have an infinite number of significant digits.

Two types of exact numbers are common:

1. Numbers obtained by counting things that can be done with certainty and that are not divisible. For example:

• For example, there are 13 people taking ASTR3130 (one would not say there are 13.000 people in the class, or 13.078!)

• Or, there are 6 types of quark (not 6.00).

2. Numbers derived by definition. For example:

• 1 m = 100 cm.

• 12 objects are in a dozen.

• 12 inches are in a foot.

Because these numbers have an infinite number of significant digits, we ignore these as a limiting factor in determining the number of significant figures in the result of a calculation (see below).

Most (but not all) exact numbers are integers, and it is common in the scientific literature to actually spell out such exact numbers when they are less than a certain value (commonly, 20 = "twenty"). For example:

• "We undertook three separate tests...", not "We undertook 3 separate tests...".

• "The measurement was made by eleven students.", not "The measurement was made by 11 students."

### Rules for Retaining Sigificant Figures in Mathematical Calculations

DON'T SIMPLY COPY YOUR CALCULATOR OUTPUT!

When carrying out calculations, the general rule is that the accuracy of a calculated result is limited by the least accurate measurement involved in the calculation.

This then makes very clear rules for how many digits to report after the calculation:

1. Addition and Subtraction: Add or subtract in the normal fashion. Then round the answer to the LEAST number of decimal places as offered in any number in the problem. For example:

• 100 (assume 3 significant digits) + 23.643 (5 significant digits) = 123.643, but this should be rounded to 124 (3 significant digits).

• 20.63 + 6.6 + 3.786 = 31.016, but this should be stated as (rounded to) 31.0.

• 145.67 - 98.432 = 47.238, but this should be stated as (rounded to) 46.24.

2. Multiplication and Division: The LEAST number of significant figures in any number in the calculation determines the number of significant figures in the answer. For example:

• 3.0 (2 significant figures) x 12.60 (4 significant figures) = 37.8000, which should be rounded to 38 (2 significant figures).

• 30.2 m x 2.3 m = 69.46 m2 = 69 m2

• 124 cm / 41 s = 3.0 cm/s

### Rules for Rounding Off Numbers

1. If the digit to be dropped is greater than 5, the last retained digit is increased by one. For example,

• 12.6 is rounded to 13.

• 559 is rounded to 560.

2. If the digit to be dropped is less than 5, the last remaining digit is left as it is. For example,

• 12.4 is rounded to 12.

• 562 is rounded to 560.

3. If the digit to be dropped is 5, and if any digit following it is not zero, the last remaining digit is increased by one. For example,

• 12.51 is rounded to 13.

• 12.500001 is rounded to 13.

4. If the digit to be dropped is 5 and is followed only by zeroes, the last remaining digit is increased by one if it is odd, but left as it is if even. For example,

• 11.5 is rounded to 12,

• but 12.5 is also rounded to 12.

Essentially this rule means that if the digit to be dropped is 5 followed only by zeroes, the result is always rounded to the even digit.

The rationale for this is to avoid biases by how we round, because half of the time we will be rounding up, and half the time we will be rounding down.

• If you adopted a rule of always "rounding up", you will be always biasing your results to higher numbers.

### Using Calculators and When to Round

Despite my warnings above not to simply report what appears in your calculator output, it is important to be clear about when in the course of doing a calculation it is appropriate to apply the rules given above for significant digits and rounding.

• It is a good practice when using a calculator to also write down what is happening at each step in the calculation. if you work the entirety of a long calculation without writing down any intermediate results, you may not be able to tell if an error is made. On the other hand, even if you realize that an error has occurred in your calculation, you may not be able to tell at what step that error was made.

• In a long calculation involving a mix of mathematical operations, carry as many digits as practical throughout the entire set of calculations and then only round the final result according to the appropriate rules.

For example,:

(5.00 / 1.235) + 3.000 + (6.35 / 4.0) =

4.04858... + 3.000 + 1.5875 =

8.630829... =

8.6

The following rules apply to this calculation:

• The division in the first term should result in 3 significant figures (shown in red).

• The second term is clearly declared to be 4 significant terms.

• The division in the last term should result in 2 significant figures.

• The three numbers added together should result in a number that is rounded off to the last significant digit occurring furthest to the right common to all three terms, which is in the third term.

• Thus, the final result should be rounded with 1 digit after the decimal because the accuracy in the result has been limited by the accuracy of the division in the third term.

• Fortunately, many calculators allow you to carry through the results of intermediate calculations in the active display and memory when performing a complex series of calculations. This means that you can automatically carry through the results of each individual step without re-rentering it, and this allows you to retain many digits without truncation/rounding errors introduced at intermediate stages of the calculation.

• On the other hand, if you prematurely round off an intermediate result while carrying out a long calculation, you risk doing so at a digit that is too far to the left to retain the proper level of accuracy at the digit you need to keep significant by the end of the calculation.

For this reason, it is good practice to carry all digits through to the final result before rounding, which should happen at the end of the calculation.

• Note, however, if in the narrative of your lab report you are describing an intermediate result of a calculation, you should not in your text cite that number with more digits than is appropriate at that point in the calculation (i.e., too many significant digits).

A rule of thumb is to always retain as many digits as possible at all stages of a calculation, BUT if one of the intermediate results of the calculation is being discussed in a narrative, apply the rounding to that number when you are citing it, even if more digits are carried through to the final calculation.

### Now Test Yourself

NOW THAT YOU KNOW THE RULES FOR SIGNIFICANT FIGURES, PLEASE TAKE THE SELF-CORRECTING QUIZ LOCATED AT THIS WEBSITE.

Also, try the quiz here.

All material copyright © 2012 Steven R. Majewski. All rights reserved. These notes are intended for the private, noncommercial use of students enrolled in Astronomy 3130 at the University of Virginia.