How Science Works – 6 Things You Need to Know Now

I don’t have to look at posts on social media for long to conclude many people don’t have a grasp of how science really works. They don’t understand why official recommendations keep changing or why studies come up with different results.

 Most of us may remember learning about the scientific method in school.  It seems so straightforward and neat, but the application of that process is anything but neat.  A failure to realize that results in a lot of confusion and frustration which have become especially evident in the COVID-19 pandemic.

With that observation in mind, here is my humble attempt to share what I learned about how science works from some brilliant scientists who mentored me in graduate school. 

Science is Messy and Often Contradictory

The COVID-19 pandemic has generated a torrent of scientific research.  As of October 2020 more than 87,000 studies had been published. Never before have we witnessed the progression of that much research on a topic in such a short timeline.  Despite this incredible volume of research, there are still countless contradictions and unanswered questions.

With all that research, most of us expect quick, clear answers, but that is not the case. That is why people on both sides of the mask issue can point to research that supports their position. It is also why there are often mixed results on treatments and mitigation methods surrounding the pandemic.

This trait of scientific research makes it easy for those with an agenda to cherry pick studies that support their position and ignore evidence may point to the opposite conclusion. It also has led to frustration over the ever-changing recommendations from official sources.

A good example comes from a study done early in the pandemic that showed the coronavirus could live for days on some surfaces.  In light of this study, the official recommendation was to be super diligent with ways to avoid contact spread.  However, later research showed contact spread of the virus is not a significant method of transmission.

All Research Has Limitations

I can’t begin to count how many scientific studies I have read.  But I have yet to read one that has no limitations.  With research on humans, we are limited by ethics.  You could design a conclusive, controlled study on the effectiveness of wearing masks, but that would involve deliberately exposing people to the coronavirus.  Other limitations include small sample sizes, lack of controls, type of subject, unproven assumptions, and more. 

Even the “gold standard” randomized, double-blind, placebo-controlled studies have limitations.  They may not be able to control all the potential environmental variables their subjects can experience. And they will generally answer a very specific question.  For example, the subjects of research on a treatment for a disease may investigate only people who are hospitalized.  In that case, you cannot assume the treatment would work the same on people with mild symptoms of a disease. 

The Weight of Evidence

As I mentioned earlier, science is often messy and contradictory. So how do you make sense of conflicting information out there?  All you can do is look at the weight of evidence.  While scientists don’t usually try to replicate studies exactly, they will often conduct the same study and change just one variable.  Or they will look at the same question from a different angle. Study builds upon study to finally come up with a consensus on the answer to a question. 

If you look at an article in a scientific journal, the introduction of a study reviews the existing scientific literature, focusing on what previous research has failed to examine or where previous studies have produced inconsistent or contradictory results.  The scientists then design a study to find a missing piece of the puzzle.  Over time, the research builds on itself to develop a consensus on the bigger questions.  Even though there will be studies that don’t support this consensus, the lion’s share of quality research eventually points in one direction.  

My mentors called this “The Weight of Evidence”.  Always look for conclusions supported by the weight of scientific evidence. You can rarely make a broad conclusion from a single study.   So when you read a news article that says, “New study shows…” the first question you should ask is, “What do other studies show?”

Type of Research Impacts Conclusions

A big factor that affects the “Weight of Evidence” is the type and quality of the research, not just the number of studies.  In biological science, we see several general categories of research in published scientific journals.

Anecdotal Evidence

This may include case studies or just observations. Let’s say a doctor treated 20 patients with a certain protocol and they all got better within 2 weeks.  On the surface, that looks impressive, and it certainly calls for examining the treatment through other research.  But with the case study itself, we don’t know if those patients would have gotten better anyway.  They weren’t compared with patients who didn’t receive the treatment. There were no controls.  

Epidemiological Research

Epidemiology looks at populations.  These studies can involve huge numbers of subjects and track them over decades.  Periodically the subjects may be required to fill out a survey or submit to some other form of data collection for weight, cardiovascular health, metabolic health, treatment protocols, etc. 

Epidemiological research is critically important in guiding future research. While scientists can statistically correct for many variables, they can’t establish controls on environmental factors that may produce different conclusions. 

Controlled Research

There are different types of controlled studies, but we’re going to examine them as a broad category.  These studies seek to control as many variables as possible so that we can be more certain that one thing affects something else.

An example is with drug treatments.  Patients suffering from a certain disease are divided into two groups: one group gets the treatment being studied, and another group gets a placebo.  The researchers need to try to keep demographic representations as equal as possible in the two groups.  The results would be compromised if everybody who got the treatment was under 30 and everybody who got the placebo was over 60.  Another variable would be severity of illness.  You might want to conduct the study only on people who are hospitalized. 

Correlation Doesn’t Necessarily Equal Causation

As I mentioned earlier, epidemiological research looks at trends in populations.  In its favor, sample sizes can be huge.  So the scientific community will take its results seriously and develop a body of research to test the epidemiological study’s conclusions. 

But, as I mentioned earlier,  a limitation of this type of research is that there are usually uncontrolled variables.  Even if the correlation between two things is strong, something else not accounted for may influence the results.  Thus, the mantra my mentors taught me: “Correlation doesn’t necessarily equal causation.”

We Get Most of Our Science through Filters

Most people get their information about scientific advancements through the news, politicians, commentators, blogs, etc.  They don’t actually read the original research.  It’s just too time consuming, especially if you don’t have a solid background on the subject.  This opens the door for people with agendas to cherry pick studies and give you a skewed view of what the research really says.  Others manipulate data and just plain lie about the current state of the science. 

A good scientific reporter should include a link or citation to the study they are reporting on.  Take time to look at that original study. Read the conclusions of the authors. That will give you the big picture of what they found out. Then your information comes from the source. Fake science reporters may give links, but they only go to the homepage of a scientific journal and not to an actual study.  That’s a big red flag. Ignore these stories, even if they agree with your perspective on an issue.  

I’m sure there are other truths about science that I missed.  But I hope this gives you enough insight to ease the frustration and develop some patience with the scientific community as they continue to search for answers.  Science takes time.