Section: 8 | Detection of Outliers in Measurements |

How to Cite this Reference

DETECTION OF OUTLIERS IN MEASUREMENTS

Thomas J. Bruno and Paris D. N. Svoronos

The field of outlier detection and treatment is considerable, and a rigorous mathematical discussion is well beyond any treatment that is possible here. Moreover, the practice in the treatment of analytical results is usually simplified, because the number of observations is often not very large. The two most common methods used by analysts to detect outliers in measured data are versions of the Q-test (Refs. 1–3, 6) and Chauvanet’s criterion (Refs. 4–6), both of which assume that the data are sampled from a population that is normally distributed.

References

Dean, R. B., and Dixon, W. J., Anal. Chem. 23, 636, 1951. [https://doi.org/10.1021/ac60052a025]
Day, R. A., and Underwood, A.L., Quantitative Analysis, Sixth Edition, Prentice Hall, Englewood Cliffs, NJ, 1991.
Efstathiou, C. E., Dixon’s Q-test: Detection of a Single Outlier, <http://www.chem.uoa.gr/applets/AppletQtest/Text_Qtest2.htm>, Laboratory of Analytical Chemistry, Department of Chemistry, National and Kapodistrian University of Athens, 2008.
Taylor, J. R., An Introduction to Error Analysis, Second Edition, University Science Books, Sausalito, CA, 1997.
Benziger, J. B., and Aksay, I.A., Notes on Data Analysis, <http://www.princeton.edu/~che346/Notes/Analysis.pdf>, Department of Chemical Engineering, Princeton University, 1999.
Bruno, T. J., and Svoronos, P. D. N., CRC Handbook of Basic Tables for Chemical Analysis – Data-Driven Methods and Interpretation, Fourth Edition, CRC Press/Taylor & Francis, Boca Raton, FL, 2021. [https://doi.org/10.1201/b22281]

Q-Test

To perform the Q-test, one calculates the Q value given by:

Q = Q_gap/R

where Q_gap is the difference between the suspected outlier and the measured value closest to it, and R is the range of all the measured values in the data set. One then compares the calculated Q value with the critical Q values in Table 1.

Go to interactive table

Mobile-friendly table

TABLE 1. Critical Q–Test Values for Different Confidence Levels

Number of observations	Q_crit, 90% Confidence level	Q_crit, 95% Confidence level	Q_crit, 99% Confidence level
3	0.941	0.970	0.994
4	0.765	0.829	0.926
5	0.642	0.710	0.821
6	0.560	0.625	0.740
7	0.507	0.568	0.680
8	0.468	0.526	0.634
9	0.437	0.493	0.598
10	0.412	0.466	0.568

If the calculated value of Q is greater than the appropriate value of Q_crit, then the value is a suspected outlier.

Chauvenet’s Criterion

To perform Chauvenet’s test on a set of measurements, one first must calculate the mean and standard deviation of the data. Then one calculates:

τ = (x_i – x_ave)/σ

where x_i is the suspected outlier, x_ave is the mean of all the measurements, and σ is the standard deviation. One then compares the calculated value of τ with τ_crit in the following table.

Go to interactive table

Mobile-friendly table

TABLE 2. Critical Chauvenet's τ Values as a Function of the Number of Observations

Number of observations, N	τ_crit
5	1.65
6	1.73
7	1.81
8	1.86
9	1.91
10	1.96
15	2.12
20	2.24
25	2.33
50	2.57
100	2.81
150	2.93
200	3.02
500	3.29
1000	3.48

If the calculated value of τ is greater than the value of τ_crit, then the value is a suspected outlier.

For numbers of observations between those given in Table 2, especially for a large number of observations, one may use the following plot to estimate the value of Chauvenet’s τ_crit.

plot shows a curve of no. of observations vs. tau crit

Entry Display

This is where the entry will be displayed

Other ChemNetBase Products

Log Out From Your Workspace

Are you sure?

Create your personal workspace

Incorrect login details

My Account

Your Account Details

To view your account details Log In
Forgotten password?

Subscription To ChemNetBase Products

Handbook Of Chemistry and Physics Online

Enquire

Combined Chemical Dictionary

Enquire

Dictionary of Natural Products

Enquire

Dictionary of Organic Compounds

Enquire

Dictionary of Drugs

Enquire

Dictionary of Inorganic and Organometallic Compounds

Enquire

Dictionary of Commonly Cited Compounds

Enquire

Dictionary of Marine Natural Products

Enquire

Dictionary of Food Compounds

Enquire

Polymers: A Property Database

Enquire

Properties of Organic Compounds

Enquire

Change Your Workspace Password

Update your Personal Workspace Details

Workspace Log In Reminder

Searching for Chemicals and Properties

The CRC Handbook of Chemistry and Physics (HBCP) contains over 700 tables in over 450 documents which may be divided into several pages, all categorised into 17 major subject areas. The search on this page works by searching the content of each page individually, much like any web search. This provides a challenge if you want to search for multiple terms and those terms exist on different pages, or if you use a synonym/abbreviation that does not exist in the document.

We use metadata to avoid some of these issues by including certain keywords invisibly behind each table. Whilst this approach works well in many situations, like any web search it relies in the terms you have entered existing in the document with the same spelling, abbreviation etc.

Since chemical compounds and their properties are immutable, a single centralised database has been created from all chemical compounds throughout HBCP. This database contains every chemical compound and over 20 of the most common physical properties collated from each of the >700 tables. What's more, the properties can be searched numerically, including range searching, and you can even search by drawing a chemical structure. A complete list of every document table in which the compound occurs is listed, and are hyperlinked to the relevant document table.

The 'Search Chemicals' page can be found by clicking the flask icon in the navigation bar at the top of this page. For more detailed information on how to use the chemical search, including adding properties, saving searches, exporting search results and more, click the help icon in to top right of this page, next to the welcome login message.

Below is an example of a chemical entry, showing its structure, physical properties and document tables in which it appears.

Cookie Policy

We have developed this cookie policy (the “Cookie Policy”) in order to explain how we use cookies and similar technologies (together, “Cookies”) on this website (the “Website”) and to demonstrate our firm commitment to the privacy of your personal information.

The first time that you visit our Website, we notify you about our use of Cookies through a notification banner. By continuing to use the Website, you consent to our use of Cookies as described in this Cookie Policy. However, you can choose whether or not to continue accepting Cookies at any later time. Information on how to manage Cookies is set out later in this Cookie Policy.

Please note that our use of any personal information we collect about you is subject to our Privacy Policy.

What are Cookies?

Cookies are small text files containing user IDs that are automatically placed on your computer or other device by when you visit a website. The Cookies are stored by the internet browser. The browser sends the Cookies back to the website on each subsequent visit, allowing the website to recognise your computer or device. This recognition enables the website provider to observe your activity on the website, deliver a personalised, responsive service and improve the website.

Cookies can be ‘Session Cookies’ or ‘Persistent Cookies’. Session Cookies allow a website to link a series of your actions during one browser session, for example to remember the items you have added to a shopping basket. Session Cookies expire after a browser session and are therefore not stored on your computer or device afterwards. Persistent Cookies are stored on your computer or device between browser sessions and can be used when you make subsequent visits to the website, for example to remember your website preferences, such as language or font size.

Cookies We Use and Their Purpose

We use three types of Cookies - ‘Strictly Necessary’ Cookies, ‘Performance’ Cookies and ‘Functionality’ Cookies. Each type of Cookie and the purposes for which we use them are described in this section. To learn about the specific Cookies we use, please see our List of Cookies.

1. Strictly Necessary Cookies

‘Strictly Necessary’ Cookies enable you to move around the Website and use essential features. For example, if you log into the Website, we use a Cookie to keep you logged in and allow you to access restricted areas, without you having to repeatedly enter your login details. If you are registering for or purchasing a product or service, we will use Cookies to remember your information and selections, as you move through the registration or purchase process.

Strictly Necessary Cookies are necessary for our Website to provide you with a full service. If you disable them, certain essential features of the Website will not be available to you and the performance of the Website will be impeded.

2. Performance Cookies

‘Performance’ Cookies collect information about how you use our Website, for example which pages you visit and if you experience any errors. These Cookies don’t collect any information that could identify you – all the information collected is anonymous. We may use these Cookies to help us understand how you use the Website and assess how well the Website performs and how it could be improved.

3. Functionality Cookies

‘Functionality’ Cookies enable a website to provide you with specific services or a customised experience. We may use these Cookies to provide you with services such as watching a video or adding user comments. We may also use such Cookies to remember changes you make to your settings or preferences (for example, changes to text size or your choice of language or region) or offer you time-saving or personalised features.

You can control whether or not Functionality Cookies are used, but disabling them may mean we are unable to provide you with some services or features of the Website.

First and Third Party Cookies

The Cookies placed on your computer or device include ‘First Party’ Cookies, meaning Cookies that are placed there by us, or by third party service providers acting on our behalf. Where such Cookies are being managed by third parties, we only allow the third parties to use the Cookies for our purposes, as described in this Cookie Policy, and not for their own purposes.

The Cookies placed on your computer or device may also include ‘Third Party’ Cookies, meaning Cookies that are placed there by third parties. These Cookies may include third party advertisers who display adverts on our Website and/or social network providers who provide ‘like’ or ‘share’ capabilities (see the above section on Targeting or Advertising Cookies). They may also include third parties who provide video content which is embedded on our Website (such as YouTube). Please see the website terms and policies of these third parties for further information on their use of Cookies.

To learn about the specific First Party and Third Party Cookies used by our, please see our List of Cookies.

Managing Cookies

You always have a choice over whether or not to accept Cookies. When you first visit the Website and we notify you about our use of Cookies, you can choose not to consent to such use. If you continue to use the Website, you are consenting to our use of Cookies for the time being. However, you can choose not to continue accepting Cookies at any later time. In this section, we describe ways to manage Cookies, including how to disable them.

You can manage Cookies through the settings of your internet browser. You can choose to block or restrict Cookies from being placed on your computer or device. You can also review periodically review the Cookies that have been placed there and disable some or all of them.

You can learn more about how to manage Cookies on the following websites: www.allaboutcookies.org and www.youronlinechoices.com.

Please be aware that if you choose not to accept certain Cookies, it may mean we are unable to provide you with some services or features of the Website.

Changes to Cookie Policy

In order to keep up with changing legislation and best practice, we may revise this Cookie Policy at any time without notice by posting a revised version on this Website. Please check back periodically so that you are aware of any changes.

Questions or Concerns

If you have any questions or concerns about this Cookie Policy or our use of Cookies on the Website, please contact us by email to [email protected]

You can also contact the Privacy Officer for the Informa PLC group at [email protected].

Our Cookies

Accessibility

Accessibility in Handbook of Chemistry and Physics 103rd Edition

The Voluntary Product Accessibility Template (VPAT) is a self-assessment document which discloses how accessible Information and Communication Technology products are in accordance with global standards.

The VPAT disclosure templates do not guarantee product accessibility but provide transparency around the product(s) and enables direction when accessing accessibility requirements.

Taylor & Francis has chosen to complete the International version of VPAT which encompasses Section 508 (US), EN 301 549 (EU) and WCAG2.1 (Web Content Accessibility Guidelines) for its products.

Click here for more information about how to use this web application using the keyboard.

VPAT WCAG

Feedback

Top Notification Bar Dialog Header

DETECTION OF OUTLIERS IN MEASUREMENTS

References

Q-Test

TABLE 1. Critical Q–Test Values for Different Confidence Levels

Chauvenet’s Criterion