Confidence Interval

Initiation

Off[General::spell, General::spell1]
Needs["Statistics`ConfidenceIntervals`"]
Needs["Statistics`DescriptiveStatistics`"]
Needs["Statistics`MultiDescriptiveStatistics`"]
Needs["Graphics`MultipleListPlot`"]

Confidence Level

This is perhaps easist to understand from the normal or Gaussian distribution.  It is a measure of the probability that a ramdom measurement of a parameter x will be within zσ of the mean, μ, of the distribution.  This, of course, relates to the precision, not the accuracy of the measurement.

The parameter z sets the confidence level.  If z=1, the level is about 68%, i.e., within one standard deviation of the mean.  For z=2, the level is about 95%.  These can be read from Table C.2 in Bevington, which is the Integral of Gaussian Distribution, (Subscript[P, G])(x; μ, σ).  

In other words, the randaom samplings of a quantity x are distributed according to the Gaussian distribution about a mean μ with standard deviation σ, (Subscript[P, G])(x; μ, σ) gives the probability that |x - μ| < zσ.

This probability also can be calculated in Mathematica, with the cumulative distribution function, CDF[dist, x].  It is more general than Table C.2 because any distribution function can be calculated. The cumulative distribution function is used in evaluating probabilities for statistical hypotheses.  

You could have used it in HW4 2.8 b) if dist were specified as the normal distribution.  In that case, you were trying to determine if the specific sampling were part of the parent population.

({{"   z", "Int Gauss", "  Prob >"}, {1.7407681499880687`, 0.9182757730307076`, 0.08172422696929238`}})

Even though it was at the edge of the distribution, it was less than two standard deviations away.

Small Samples

Occasionally, due to one or more constraints, it is not possible to collect adequate statistics.  Thus, you have a sample that is too small and your uncertainty in your measurement is likely to be larger than what would be given by a normal distribution.  In this case, one can use the so-called Student's t-distribution, 

			(Subscript[P, t])(t,ν) = (1/(Sqrt[π v]))\(\(Γ(\(ν + 1\)\/2)\)\/\(Γ(ν\/2)\)\)(((1 + ((t^2)/ν)))^(-(ν + 1)/2)),

which takes into account the number of degrees of freedom, ν, when establishing confidence levels.  The Student's t-distribution deals with the problems associated with inference based on small samples: the calculated mean, μ, and standard deviation, σ, may by chance deviate from the real mean and standard deviation (i.e., what you'd measure if you had many more data items: a large sample). For example, it likely that the true mean is close to the mean calculated from a sample of N randomly collected samples.  For N=5, a 95% confidence level means that 95% of the time the actual mean would be in the range:  μ ± 2.776 σ/(Sqrt[N]); N = 10:   μ ± 2.262 σ/(Sqrt[N]);  N = 20: μ ± 2.093 σ/(Sqrt[N]); if N = 40; μ ± 2.023 σ/(Sqrt[N]); and for large N:  μ ± 1.960 σ/(Sqrt[N]).  

Consider the following example where we will use the Student's t-distribution to estimate the 95% confidence interval.  We note that  a 95% confidence level means that 95% of the time the actual mean would be in the given range!

Data

data1 = {2.1, 1.2, 0.7, 1.0, 1.1, 3.2, 3.2, 3.3, 2.1, 
          0.3};
data2 = {0.7, 1.0, 1.1, 3.2, 3.2};

N=10 (data1)

num1 = Length[data1]
avg1 = Mean[data1]
sig1 = (Sqrt[Variance])[data1]
MeanCI[data1]
MeanCI[data1] - avg1
2.262 sig1/(Sqrt[num1])

N=5 (data2)

num2 = Length[data2]
avg2 = Mean[data2]
sig2 = (Sqrt[Variance])[data2]
MeanCI[data2]
MeanCI[data2] - avg2
2.776 sig2/(Sqrt[num2])

Gaussian from Table C.2

1.96 sig1/(Sqrt[num1])
1.96 sig2/(Sqrt[num2])

Linear Correlation

Linear correlation test allows you to determine if two parameters are linearly related.  The correlation coefficient, r, is given by:

r=(((∑_i)\(\((x\_i - x\&_)\) \((y\_i - y\&_)\)\))/((Sqrt[(∑_i)\(\((x\_i - x\&_)\)\^2\)])(Sqrt[(∑_i)\(\((y\_i - y\&_)\)\^2\)])))" "

When r is ~ ± 1, the two parameters are said to be correlated.  If r ~ 0, there is good reason to doubt any significant correlation.  Consider your linear fit from prob HW3:

xlist = {2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 22, 
          24};
ylist = {6.9, 18.8, 26.8, 39.2, 45.5, 53.7, 68.5, 72.2, 82.1, 93.7, 
          104.6, 114.3};
MultipleListPlot[Transpose[{xlist, ylist}], PlotRange -> All, 
      Frame -> True, Axes -> False, 
      FrameLabel -> {"x", "y"}, ImageSize -> 450, 
      TextStyle -> {FontFamily -> "Times", 
          FontWeight -> "Bold", FontSize -> 15}, 
      PlotJoined -> False, SymbolShape -> {PlotSymbol[Box, 3]}, 
      PlotLabel -> "Linear fit"]
num3 = Length[xlist]
r = Correlation[xlist, ylist]

zlist = {57.7, 68.8, 69.6, 58.1, 80.8, 69.6, 48.5, 55.8, 70.8, 59.6, 
          66.9, 52.3};
MultipleListPlot[Transpose[{xlist, zlist}], 
      PlotRange -> {{0, 25}, {0, 100}}, Frame -> True, 
      Axes -> False, FrameLabel -> {"x", "y"}, 
      ImageSize -> 450, 
      TextStyle -> {FontFamily -> "Times", 
          FontWeight -> "Bold", FontSize -> 15}, 
      PlotJoined -> False, SymbolShape -> {PlotSymbol[Box, 3]}, 
      PlotLabel -> "Linear fit"]
num3
r = Correlation[xlist, zlist]

It is pretty clear that the parameters are correlated in the first example.  What about the second example.  We can find the probability that the observed data could come for an uncorrelated parent population.  Such a probability can be determined from Table C.3 or Fig. C.3.  In the first example, the probability that the observed date is due to an uncorrelated parent population is much less than 0.1%.  In the second case, however, it is close to 50%.

This can be generalized to multivariable correlations as well.

F Test

f = \(χ\_1\%2\/χ\_2\%2\)((Subscript[ν, 2])/(Subscript[ν, 1]))

The F-ratio distribution is the distribution of the ratio of two chi-square variables divided by their respective degrees of freedom.  It is commonly used when comparing the variances of two populations in hypothesis testing.