this formula is unstable and is not the right way to do it. interestingly, there is a simpler method that always gives superior results.
| a lot of calculators make use of the following textbook
formula for calculation of standard deviation.
this formula is unstable and is not the right way to do it. interestingly, there is a simpler method that always gives superior results. |
|
the problem in the above is that differences squared require more precision than ordinary differences. naive use of the above can even lead to the square root of a negative number.
in a measurement experiment, samples are recorded. those samples will be about a mean value. if this mean value is considerably different from zero, this is when problems occur. for example, suppose my measurements are x1=1,000,000 x2 = 999,999 and x3 = 1,000,001. it is clear that my mean is 1,000,000 and the standard deviation should be 1. however, plug these numbers into most calculators and the result will be zero!.
the problem is that the calculator needed 12 digits to square 10^6 and subtract. it only has 10, so the difference drops off the end of the calculation yielding zero.
one way to correct for this is to make two passes of the data. if the data are adjusted by subtracting the mean value, the variance is unchanged and so the above problem is corrected.
the problem with this approach for calculators is that it involves storing the dataset, rather than keeping a tally of values.
calculators that do this and therefore get answers correct include the hp28c and hp48g.
but amazingly, there is a simpler method than the textbook formula that is stable and does not require keeping record of the dataset.
here is the method:
to extract the standard deviation after n datum, calculate sqrt(s/(n-1))
the only calculator ive seen to do this is the hp71b.