March 19, 2023
by Jesse Harris, Digital Marketing Coordinator, ACD/Labs
Is there anything that analytical chemists hate more than false positives and false negatives? Your equipment was operating correctly, you ran your experiment exactly to script, but your result turned out to be an error. We love certainty and accuracy as scientists, but sometimes problems occur despite our best efforts.
While it is impossible to eliminate errors completely, it is possible to control them. Understanding the relationship between false positives and negatives allows us to optimize our experimental design. To begin with, what are false positives and false negatives?
What are False Positives and False Negatives?
False positives and false negatives are types of inaccurate results. A false positive is getting a positive result that should be negative (this is also known as a “type I error”). A false negative is getting a negative result that should be positive (this is also known as a “type II error”).
Let’s consider an example to understand these in practice. Imagine you need to determine if a specific toxic chemical is present in drinking water. The test is not 100% accurate. In that case, there are four possible outcomes:
| Test Result | Chemical Present | Chemical Not Present |
| Positive Result | True Positive | False Positive |
| Negative Result | False Negative | True Negative |
Both error types can cause problems. In the case of the drinking water, the false positive means you raise an unnecessary alarm about contamination. A false negative would mean the toxic chemical would enter the water supply undetected.
Of course, you can’t tell if a results is “true” or “false”. You just see the results, and need to make the interpretation. For our water samples, your test either tells you “yes there is a toxin,” or “no, there is no toxin.” There is no additional note to tell you when the results are wrong.
So how do we stop false positives and false negatives? Unfortunately, the two are often connected. For the water testing example, the number of false positives vs. negatives might be related to the concentration of your samples. Concentrating your sample will likely decrease the chance of a false negative but increase the risk of a false positive. Diluting samples will do the opposite.
Adjusting the balance between false positives and negatives might be sufficient to satisfy you. For water testing, a false positive is inconvenient, while a false negative could be extremely dangerous. In that case, you may want to design a test that reduces the chance of a false negative as low as is practical.
But what if we want to reduce overall rates of error? How do we escape this trade-off? Luckily, you have some options.
Improve Your Method
The most effective way to reduce both your false positives and negatives is using a high-quality method. This is particularly important in chromatography, though method development work is necessary in other analytical techniques. Methods are handed down from one scientist to another, sometimes with little training or explanation. These methods may not be optimized for the work you are currently doing.
Why do researchers use underperforming methods? There are many reasons, but one of the most important is because method development is time-consuming. Scientists often use trial-and-error rather than a rational approach when optimizing their methods. You could spend weeks refining a method, only to find you have made no progress.
Software can significantly reduce your method development time. For example, AutoChrom allows chromatographers to predict separation times for compounds under various conditions. Not only is this more efficient, but it also enables researchers to optimize a broader range of variables then could realistically be achieved in the lab.
Imad Haider Ahmad, PhD., Associate Principal Scientist at Merck, offered a short demonstration of this software:
Use Multiple Methods
You’ve spent the time to develop the best method possible, but you are still getting too many false positives or false negatives. What do you do next? Using a second analytical method will increase your accuracy even more.
Let’s return to our example of a chemical contaminant in wastewater. This is what the table looks like after a secondary test.
| Test 1 Result | Test 2 Result | Chemical Present | Chemical Not Present |
| Test 1 Positive | Test 2 Positive | True Positive | False Positive |
| Test 2 Negative | Disagreement | Disagreement | |
| Test 2 Negative | Test 2 Positive | Disagreement | Disagreement |
| Test 2 Negative | False Negative | True Negative |
Additional analysis would be needed in cases where the tests disagree. This does increase the workload of testing, but it will significantly reduce the number of errors. A researcher that uses a single test that was 95% accurate would have a 5% error rate. By using two tests that are both 95% accurate, the error rate drops to only 0.25%! This applies to the rate of both false positives and false negatives.
You can further improve your performance by choosing a secondary method that specifically targets “blind spots.” Each analytical technique has certain compounds where it is extremely sensitive, while others it is less useful for. For example:
- UV spectrometry is very sensitive to aromatic compounds.
- IR spectrometry is sensitive to ketones and aldehydes.
- NMR can be used to test for the presence of specific heteroatoms, such as phosphorous, fluorine, and nitrogen.
If you know the cause of your issue, you can pick a method (or combination of methods) that addresses the problem you are trying to solve. For our water contamination example, you might be able to use NMR to confirm the presence of a toxin with a flourine atom, or to test for compounds known to cause false positives that happens to contain a flourine.
But what about the time investment? Technology can help here too. A 2012 American Laboratory article looked at the time and monetary impact of using Automated Structure Verification (ASV). This software uses NMR and LC-MS data to identify the compounds. Such a system saves scientists time in structure verification as long as false negatives and false positives fall within an acceptable range. The study found that ASV would reduce human error by an estimated 78% with a total error (both false positive and false negative) of the system being ~5%. The authors also estimated that ASV would save money for organizations with 30 or more researchers.
Read the ASV study here: Part 1, Part 2
Understand Causes of Error in Analytical Chemistry
Every experiment has its nuances. Errors can result from random chance, but they often have an identifiable cause. That cause could be the limits of your analytical technique, impurities in your samples, or data analysis issues. If you understand the problem, you can either solve it or work around it.
Imagine you were screening plant material for a specific natural product. Your test is frequently returning false positives. Biological samples are complex and often closely related compounds appear in the same plant. Tinkering with your analytical method might be a waste of time; you should instead improve your sample preparation.
You should also know your method’s limit of detection (LOD) and quantification (LOQ). These are the lowest concentration at which a method can accurately detect or quantify the presence of a specific compound. Tests conducted below the LOD/LOQ are going to be highly inaccurate. This may sound obvious, but scientists often inherit methods they didn’t develop, and may not realize that small changes in experimental conditions have led to concentrations dropping under the LOD/LOQ.
The best way to know if your system has unique challenges is to talk with people who have experience doing the experiments you are planning. A 10-minute conversation can save you 10 weeks of work!
Acceptable Levels of False Positives and False Negatives
Unfortunately, there is always a chance of error. Scientists love certainty, but that is not how the world works. If 100% accuracy isn’t possible, what is the highest level you can practically achieve?
You are also limited by time. You could use top-quality standards before every test, run everything in triplicate on multiple state-of-the-art instruments, as well as send samples to a third-party lab for back-up verification. Your accuracy will be incredible, but your project will also be incredibly time-consuming and expensive.
The real trade-off isn’t between false positives and false negatives but between accuracy and efficiency. Experienced scientists know how to identify the importance of each experiment and allocate resources to reflect that. At a certain point, all you can do is hope your results are consistent enough that you can trust them.
About the Author
