How to Distinguish Good Research from Bad

I've forgotten who the lecturer was, but the person presented the following two BBC articles:

Same newspaper, same year, same topic, but two vastly different conclusions. Both cannot be true at the same time.

Good versus bad research

Distinguishing good from bad research is no simple exercise and will always involve subjective assessments. When researchers go thorough, they use the GRADE approach.

I believe checking the following two things will make you much better equipped:

• How large the study is. Studies with few participants reporting large effects are reason for skepticism.
• How well researchers controlled for confounding factors. The evidence hierarchy can serve as a guide here.

When we want to say something about causal relationships, we trust the bottom least and the top most.

Systematic reviews

Systematic reviews are summaries of all available research within an area. They can include meta-analyses that combine results from multiple studies for more robust results.

Randomized controlled trials

RCTs examine intervention effects by randomly assigning participants to groups. This is the gold standard for causal relationships.

Non-randomized studies

These compare groups without random assignment, making it harder to interpret differences.

Observational studies

Researchers only observe participants. Many confounding factors are possible.

Case studies

Surveys of individual experiences with small numbers and subjective interpretations.

Expert opinions

Statements from experts. Should be considered but with awareness of potential biases.

More than five a day?

The article concluding more than five a day probably doesn't reduce mortality is a systematic review. The other is observational. Trust the first more.

This post is a rewritten excerpt from my book, "Better Decisions."