1.16: Detection of Gross Error: The Q Test
When one or more data points appear far from the rest of the data, there is a need to determine whether they are outliers and whether they should be eliminated from the data set to ensure an accurate representation of the measured value. In many cases, outliers arise from gross errors (or human errors) and do not accurately reflect the underlying phenomenon. In some cases, however, these apparent outliers reflect true phenomenological differences. In these cases, we can use statistical methods to help determine whether to retain the outlier. A statistical method that can help us retain or reject outliers is called the Q-test. To perform the Q-test, we first arrange the values in a data set in order of increasing value. Then, we calculate the Q value by taking the ratio of the absolute difference between a data point and its adjacent data point. This Q value is then compared with the tabulated critical Q value at a chosen significance level and appropriate degrees of freedom. If the Q value equals or exceeds the reference Q value in the table, the data point is considered an outlier and therefore rejected from the data set. Here, it is reasonable to disregard the data point as an outlier because the magnitude of the deviation cannot be logically accounted for by random (indeterminate) errors. On the other hand, if the Q value is smaller than the reference value in the table, the data point should be retained, and the interpretation is that the difference between this data point and the rest of the data is within reasonable (statistical) expectation.
