by Daniel Hartl
This month, Dan Hartl, the Higgins Professor of Biology at Harvard University, shares his thoughts on the scientific process, explaining how it is more complex (and more interesting) than it is commonly portrayed.
The “scientific method” taught in most textbooks begins with observations carried out in the real world, the formulation of an explanatory hypothesis to explain these observations, and the deduction of ideally unique predictions from this hypothesis. The predictions are tested by experiments or further observations, and the outcomes of these activities either validate the predictions or else falsify them. Validation affords evidence favoring the hypothesis, and falsification causes the hypothesis to be modified or rejected.
This modern description of the scientific method is often credited to Charles Sanders Peirce (1878) and Karl Popper (1935). The approach itself is exemplified in the work of some of the greatest scientists in history, including Galileo, Newton, Priestly, Lavoisier, Mendel, and Darwin. The progression from observation to hypothesis to prediction to experiment is logical and yet simple enough that we can teach it to sixth graders.
We also teach the scientific method in high school and college, but at this level we should be more realistic about how science works. Scientists are human beings with the same foibles and failings as everyone else. Ideally scientists ought to be objective and ignore their own personal feelings and preferences, but true objectivity is about as rare as an ivory-billed woodpecker.
In testing the predictions of any hypothesis, some experiments are more telling than others, and what evidence is considered decisive depends on the field of study. In deciding which experiments to perform, your experience and judgment come into play. You may be tempted to pitch yourself a soft ball and choose those whose results are predicted by your favorite hypothesis, even if also by alternative hypotheses. Then you can be pretty sure that the results will be “consistent” with your hypothesis, allowing you to publish instead of perish.
In other cases, experiments or observations that require sophisticated equipment can be technically challenging, and results that do not fit with expectation may be dismissed as errors due to faulty equipment or improper adjustment. Most experiments and observations yield numerical data that must be analyzed in some sort of statistical framework. This makes it all too easy to disregard some data points as being “outliers” or unrepresentative of the data as a whole.
Notoriously, some studies search for correlations between one set of observations and another, and if a researcher looks at enough of these data sets, sure enough some of the correlations are “statistically significant,” but wholly because of chance. In fact, the majority of published reports from association studies of this kind are known to be wrong (Ioannidis 2005).
Then, too, modern scientists usually work in teams or groups, and the group must decide by consensus on the critical experiments, agree on the technical accuracy of the results, and also agree on their interpretation. Once they have convinced themselves, they must then try to convince their peer scientists.
By virtue of their training, scientists are skeptical. They should be skeptical even of their own data, but, being human, their skepticism is usually sharpest when they evaluate the data of others. Peer scientists may reanalyze the data and claim that it is actually not so compelling as advertised, or they may accept the data at face value but suggest alternative hypotheses to explain it. But if the research is on the right track, eventually others will repeat the experiments or observations and confirm the results, and still others will conceive and carry out novel experiments that also support the hypothesis.
The time may come when the majority of scientists agree with your hypothesis, but don’t expect everyone to climb aboard. It is extremely rare to get everyone to agree, especially those peers who have a financial or emotional stake in the argument. Scientist can have conflicts of interest like everyone else. It takes courage to go against the interest of one’s funders, and scientists can be mulishly stubborn in admitting when their results are wrong or wrongly interpreted.
Let’s now consider what happens if your hypothesis becomes widely (though not universally) accepted. It has become a theory, and your efforts to understand how the world works have been rewarded. Is it time to break out the champagne? Not so fast. If your theory has wider social or economic implications, you then have the larger society to deal with. And here is where emotional and financial and political stakes may put up fierce resistance even when the evidence is overwhelming.
To find examples, all you have to do is follow the daily news on attitudes toward, for example, evolution, genetically modified organisms, childhood vaccination, or climate change. People who prefer things the way they are will bring up all the difficulties and pitfalls of hypothesis testing. They will enlist the small number of scientists who disagree with your theory to declare your theory “controversial” and not yet proven, even if the critics have conflicts of interest. They will argue that the widespread agreement about your theory among peer scientists only goes to prove a wide conspiracy. They will impugn your personal motives, too, saying that you are a zealot, or that you crave honors or celebrity or financial riches, or that your funding sources are tainted.
The scientific method of observation, hypothesis, prediction, and experiment or further observation is an effective method for learning how the world works. To do it right, you first have to make sure you’re not fooling yourself, then once you have convinced yourself, you need to convince your research team or group, then peer scientists, and finally the society at large.
None of this is easy, nor should it be easy. The obstacles help to weed out faulty conclusions and to separate bad science from good. Acceptance should not be easy, especially when the consequences of being wrong are high, but humanity loses when opponents who are self-interested or misinformed make acceptance impossible.
© Science Whys, 2015