An Investigation of the Effect of the Swamping Phenomenon on Several Block Procedures for Multiple Outliers in Univariate Samples

Author(s)
Thomas W. Woolley

Affiliation(s)

Department of Economics, Finance and Quantitative Analysis, Brock School of Business, Samford University, Birmingham, Alabama, USA.

Department of Economics, Finance and Quantitative Analysis, Brock School of Business, Samford University, Birmingham, Alabama, USA.

ABSTRACT

In its broadest sense, this paper reviews the general outlier problem, the means available for addressing the discordancy (or lack thereof) of an outlier (or outliers), and possible strategies for dealing with them. Two alternate approaches to the multiple outlier problem, consecutive and block testing, and their respective inherent weaknesses, masking and swamping, are discussed. In addition, the relative susceptibility of several tests for outliers in normal samples to the swamping phenomena is reported.

Cite this paper

T. Woolley, "An Investigation of the Effect of the Swamping Phenomenon on Several Block Procedures for Multiple Outliers in Univariate Samples,"*Open Journal of Statistics*, Vol. 3 No. 5, 2013, pp. 229-304. doi: 10.4236/ojs.2013.35035.

T. Woolley, "An Investigation of the Effect of the Swamping Phenomenon on Several Block Procedures for Multiple Outliers in Univariate Samples,"

References

[1] F. Wilcoxon, “Personal Communication to J. K. Brewer in Class Presentation for Statistics 405,” Florida State University, Tallahassee, 1962.

[2] V. Barnett, “The Study of Outliers: Purpose and Model,” Applied Statistics, Vol. 27, No. 3, 1978, pp. 242-250. http://dx.doi.org/10.2307/2347159

[3] V. Barnett and T. Lewis, “Outliers in Statistical Data,” 3rd Edition, Wiley, New York, 1994.

[4] W. J. Dixon, “Ratios Involving Extreme Values,” Annals of Mathematical Statistics, Vol. 22, No. 1, 1951, pp. 6878. http://dx.doi.org/10.1214/aoms/1177729693

[5] J. O. Irwin, “On a Criterion for the Rejection of Outlying Observations,” Biometrika, Vol. 17, No. 3-4, 1925, pp. 238-250. http://dx.doi.org/10.2307/2332079

[6] H. A. David, H. O. Hartley and E. S. Pearson, “The Distribution of the Ratio, in a Single Normal Sample, of Range to Standard Deviation,” Biometrika, Vol. 41, No. 3-4, 1954, pp. 482-493.

http://dx.doi.org/10.2307/2332728

[7] E. S. Pearson and M. A. Stephens, “The Ratio of Range to Standard Deviation in the Same Normal Sample,” Biometrika, Vol. 51, No. 3-4, 1964, pp. 484-487. http://dx.doi.org/10.2307/2334155

[8] F. E. Grubbs, “Sample Criteria for Testing Outlying Observations,” Annals of Mathematical Statistics, Vol. 21, No. 1, 1950, pp. 27-58. http://dx.doi.org/10.1214/aoms/1177729885

[9] T. S. Ferguson, “On the Rejection of Outliers,” Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability, Vol. 1, 1961, pp. 377-381.

[10] T. S. Ferguson, “Rules for the Rejection of Outliers,” Revue Institute Internationale de Statistica, Vol. 29, No. 3, 1961, pp. 29-43.

[11] B. Epstein, “Tests for the Validity of the Assumption That the Underlying Distribution of Life Is Exponential: Part I,” Technometrics, Vol. 2, No. 1, 1960, pp. 83-101.

http://dx.doi.org/10.2307/1266533

[12] B. Epstein, “Tests for the Validity of the Assumption That the Underlying Distribution of Life Is Exponential: Part II,” Technometrics, Vol. 2, No. 1, 1960, pp. 167-183.

http://dx.doi.org/10.2307/1266533

[13] T. Lewis and N. R. J. Fieller, “A Recursive Algorithm for Null Distributions for Outliers: I, Gamma Samples,” Technometrics, Vol. 21, No. 3, 1978, pp. 371-376.

http://dx.doi.org/10.1080/00401706.1979.1048 9784

[14] J. Likes, “Distribution of Dixon’s Statistics in the Case of an Exponential Population,” Metrika, Vol. 11, No. 1, 1960, pp. 46-54. http://dx.doi.org/10.1007/BF02613574

[15] S. S. Shapiro and M. B. Wilk, “An Analysis of Variance Test for Normality (Complete Samples),” Biometrika, Vol. 52, No. 3-4, 1966, pp. 591-611. http://dx.doi.org/10.2307/2333709

[16] S. S. Shapiro and M. B. Wilk, “An Analysis of Variance Test for the Exponential Distribution (Complete Samples),” Technometrics, Vol. 14, No. 2, 1972, pp. 355-370.

http://dx.doi.org/10.1080/00401706.1972.1048 8921

[17] S. S. Shapiro, M. B. Wilk and M. J. Chen, “A Comparative Study of Various Tests for Normality,” Journal of the American Statistical Association, Vol. 63, No. 324, 1968, pp. 1343-1372.

http://dx.doi.org/10.1080/01621459.1968.1048 0932

[18] P. L. Fisher, “Comment on the Subjective Decisions Required of the Researcher in the Selection of a Statistical Outlier Test,” Florida Journal of Educational Research, Vol. 12, 1980, pp. 27-41.

[19] P. L. Fisher, “An Investigation of Outlier Definition and the Impact of the Masking Phenomenon on Several Statistical Outlier Tests,” Unpublished Doctoral Dissertation, The Florida State University, Tallahassee, 1980.

[20] T. J. Sweeting, “Independent Scale-Free Spacing for the Exponential and Uniform Distributions,” Statistics and Probability Letters, Vol. 1, No. 3, 1983, pp. 115-119. http://dx.doi.org/10.1016/0167-7152(83)90057 -3

[21] T. J. Sweeting, “Asymptotically Independent Scale-free Spacings with Applications to Discordancy Testing,” Annals of Statistics, Vol. 14, 1986, pp. 1485-1496.

http://dx.doi.org/10.1214/aos/1176350171

[22] V. Barnett and T. Lewis, “Outliers in Statistical Data,” 2nd Edition, Wiley, New York, 1984.

[23] W. J. Dixon, “Analysis of Extreme Values,” Annals of Mathematical Statistics, Vol. 21, No. 4, 1950, pp. 488506. http://dx.doi.org/10.1214/aoms/1177729747

[24] R. B. Murphy, “On Tests for Outlying Observations,” Unpublished Doctoral Dissertation, Princeton University, Dissertation Abstracts International, 15/03, University Microfilms No. A55-534, 1951.

[25] G. L. Tietjen and R. H. Moore, “Some Grubbs-Type Statistics for the Detection of Several Outliers,” Technometrics, Vol. 14, No. 3, 1972, pp. 583-597.

[26] B. Rosner, “On the Detection of Many Outliers,” Technometrics, Vol. 17, No. 2, 1975, pp. 221-227. http://dx.doi.org/10.2307/1268354

[1] F. Wilcoxon, “Personal Communication to J. K. Brewer in Class Presentation for Statistics 405,” Florida State University, Tallahassee, 1962.

[2] V. Barnett, “The Study of Outliers: Purpose and Model,” Applied Statistics, Vol. 27, No. 3, 1978, pp. 242-250. http://dx.doi.org/10.2307/2347159

[3] V. Barnett and T. Lewis, “Outliers in Statistical Data,” 3rd Edition, Wiley, New York, 1994.

[4] W. J. Dixon, “Ratios Involving Extreme Values,” Annals of Mathematical Statistics, Vol. 22, No. 1, 1951, pp. 6878. http://dx.doi.org/10.1214/aoms/1177729693

[5] J. O. Irwin, “On a Criterion for the Rejection of Outlying Observations,” Biometrika, Vol. 17, No. 3-4, 1925, pp. 238-250. http://dx.doi.org/10.2307/2332079

[6] H. A. David, H. O. Hartley and E. S. Pearson, “The Distribution of the Ratio, in a Single Normal Sample, of Range to Standard Deviation,” Biometrika, Vol. 41, No. 3-4, 1954, pp. 482-493.

http://dx.doi.org/10.2307/2332728

[7] E. S. Pearson and M. A. Stephens, “The Ratio of Range to Standard Deviation in the Same Normal Sample,” Biometrika, Vol. 51, No. 3-4, 1964, pp. 484-487. http://dx.doi.org/10.2307/2334155

[8] F. E. Grubbs, “Sample Criteria for Testing Outlying Observations,” Annals of Mathematical Statistics, Vol. 21, No. 1, 1950, pp. 27-58. http://dx.doi.org/10.1214/aoms/1177729885

[9] T. S. Ferguson, “On the Rejection of Outliers,” Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability, Vol. 1, 1961, pp. 377-381.

[10] T. S. Ferguson, “Rules for the Rejection of Outliers,” Revue Institute Internationale de Statistica, Vol. 29, No. 3, 1961, pp. 29-43.

[11] B. Epstein, “Tests for the Validity of the Assumption That the Underlying Distribution of Life Is Exponential: Part I,” Technometrics, Vol. 2, No. 1, 1960, pp. 83-101.

http://dx.doi.org/10.2307/1266533

[12] B. Epstein, “Tests for the Validity of the Assumption That the Underlying Distribution of Life Is Exponential: Part II,” Technometrics, Vol. 2, No. 1, 1960, pp. 167-183.

http://dx.doi.org/10.2307/1266533

[13] T. Lewis and N. R. J. Fieller, “A Recursive Algorithm for Null Distributions for Outliers: I, Gamma Samples,” Technometrics, Vol. 21, No. 3, 1978, pp. 371-376.

http://dx.doi.org/10.1080/00401706.1979.1048 9784

[14] J. Likes, “Distribution of Dixon’s Statistics in the Case of an Exponential Population,” Metrika, Vol. 11, No. 1, 1960, pp. 46-54. http://dx.doi.org/10.1007/BF02613574

[15] S. S. Shapiro and M. B. Wilk, “An Analysis of Variance Test for Normality (Complete Samples),” Biometrika, Vol. 52, No. 3-4, 1966, pp. 591-611. http://dx.doi.org/10.2307/2333709

[16] S. S. Shapiro and M. B. Wilk, “An Analysis of Variance Test for the Exponential Distribution (Complete Samples),” Technometrics, Vol. 14, No. 2, 1972, pp. 355-370.

http://dx.doi.org/10.1080/00401706.1972.1048 8921

[17] S. S. Shapiro, M. B. Wilk and M. J. Chen, “A Comparative Study of Various Tests for Normality,” Journal of the American Statistical Association, Vol. 63, No. 324, 1968, pp. 1343-1372.

http://dx.doi.org/10.1080/01621459.1968.1048 0932

[18] P. L. Fisher, “Comment on the Subjective Decisions Required of the Researcher in the Selection of a Statistical Outlier Test,” Florida Journal of Educational Research, Vol. 12, 1980, pp. 27-41.

[19] P. L. Fisher, “An Investigation of Outlier Definition and the Impact of the Masking Phenomenon on Several Statistical Outlier Tests,” Unpublished Doctoral Dissertation, The Florida State University, Tallahassee, 1980.

[20] T. J. Sweeting, “Independent Scale-Free Spacing for the Exponential and Uniform Distributions,” Statistics and Probability Letters, Vol. 1, No. 3, 1983, pp. 115-119. http://dx.doi.org/10.1016/0167-7152(83)90057 -3

[21] T. J. Sweeting, “Asymptotically Independent Scale-free Spacings with Applications to Discordancy Testing,” Annals of Statistics, Vol. 14, 1986, pp. 1485-1496.

http://dx.doi.org/10.1214/aos/1176350171

[22] V. Barnett and T. Lewis, “Outliers in Statistical Data,” 2nd Edition, Wiley, New York, 1984.

[23] W. J. Dixon, “Analysis of Extreme Values,” Annals of Mathematical Statistics, Vol. 21, No. 4, 1950, pp. 488506. http://dx.doi.org/10.1214/aoms/1177729747

[24] R. B. Murphy, “On Tests for Outlying Observations,” Unpublished Doctoral Dissertation, Princeton University, Dissertation Abstracts International, 15/03, University Microfilms No. A55-534, 1951.

[25] G. L. Tietjen and R. H. Moore, “Some Grubbs-Type Statistics for the Detection of Several Outliers,” Technometrics, Vol. 14, No. 3, 1972, pp. 583-597.

[26] B. Rosner, “On the Detection of Many Outliers,” Technometrics, Vol. 17, No. 2, 1975, pp. 221-227. http://dx.doi.org/10.2307/1268354