Understanding science

Whether a nutrition claim is true can rarely be answered with a single study. This page explains the terms and ways of thinking we use in all fact checks.

How studies work

Nutrition research uses very different types of studies. They answer different questions and are prone to different errors.

• Randomised controlled trial (RCT): participants are assigned to groups by chance, for example “more eggs” and “no eggs”. Because of randomisation, the groups differ on average only in what is being tested. RCTs are therefore best at showing cause and effect. In nutrition, however, they are often short and small, and they usually measure blood values rather than diseases.
• Cohort study: many people are followed for years. They are asked what they eat, and illnesses are counted later. Such studies are large and long, but people chose their diet themselves. People who eat a lot of a particular food often differ in many other ways too.
• Case-control study: people with a disease are compared with healthy people and asked about their past diet. This is prone to recall errors.
• Cross-sectional study: a snapshot. It shows associations, but not what came first.
• Measurement or residue study: food samples are analysed for substances. It shows what was found in particular samples at a particular time and place, but not automatically whether people become ill from it.
• Systematic review and meta-analysis: all suitable studies on a question are searched for and assessed according to fixed rules and, where possible, combined statistically.
• Animal and cell studies: they explain possible mechanisms but do not allow direct conclusions about human health.

Why correlation does not prove causation

When two things occur together, it does not mean that one causes the other. A well-known example: on days when a lot of ice cream is sold, more people drown. Ice cream does not cause drowning. Both depend on warm weather.

In nutrition studies, such background factors are called confounders. People who drink a lot of soft drinks, for example, may also smoke more often or exercise less. Good studies adjust for such factors, but never completely. What remains is called residual confounding.

A second problem is reverse causation: people who are already ill often change their diet. It can then look as if the new diet caused the illness.

An association becomes more credible if it is strong, appears in many independent studies, increases with the amount consumed (dose–response relationship), fits in time, is biologically plausible and is confirmed in experiments. None of these points alone is proof.

What meta-analyses can and cannot do

A meta-analysis combines the results of many studies statistically. This makes the result more precise and shows whether studies point in the same direction.

However, a meta-analysis is only as good as the studies it contains. If it pools many biased observational studies, the overall result is biased too, just with less uncertainty. Heterogeneity also matters: if individual studies differ greatly, a single average value means little.

Several meta-analyses on the same topic often contain the same studies. Three meta-analyses are therefore not automatically three independent confirmations. We check for this overlap and do not count it more than once.

What confidence intervals mean

Studies often report results like this: “hazard ratio 1.06 (95% confidence interval 1.03–1.10)”. The first value is the best estimate: a 6% higher relative risk. The interval in brackets shows the range in which the true value very probably lies, provided the study has no systematic errors.

• If the value 1.0 (“no difference”) lies within the interval, the finding is not statistically established.
• A narrow interval means a precise estimate; a wide interval means an uncertain one.
• Statistically significant does not automatically mean important. A very small effect can be “significant” in a huge study and still hardly matter in everyday life.
• The interval only captures chance, not confounding or measurement error.

Why single studies can mislead

Any single study can reach a result that is not confirmed later, because of chance, small samples, many questions tested at once, or a special population. On top of this comes publication bias: striking results are more likely to be published than unremarkable ones.

We therefore do not assess whether some study supports a claim. With a selective choice, almost any claim can be “proven” with studies. What matters is the totality of good evidence.

Absolute and relative risks

A relative risk says by what percentage a risk rises or falls compared with another group. An absolute risk says how many people are actually affected.

Worked example (invented numbers for illustration): Of 1,000 people, 2 develop a certain disease within ten years. In another group it is 3 out of 1,000. The relative risk is 50% higher. But the absolute risk rises by only 1 in 1,000 people, i.e. by 0.1 percentage points.

Both statements are correct but feel very different. Where available, we therefore give both. If a study does not report absolute numbers, we say so.

Detection is not the same as danger

Modern laboratories can detect tiny amounts of substances, often in the range of billionths of a gram. The fact that a substance is found says nothing yet about whether it is harmful in that amount. What matters is:

• The amount actually taken in (exposure).
• Health-based guidance values, such as the tolerable weekly intake (TWI) or the acceptable daily intake (ADI). Authorities such as the European Food Safety Authority (EFSA) set them with safety margins.
• Maximum residue levels (MRLs): legal upper limits. Exceeding one is a legal violation but does not automatically mean a health hazard. Conversely, a value below the limit is not meaningless for every person and every combination of substances.
• The context: a measurement from a contamination incident, a mining area or another country does not automatically describe the normal food supply in Germany or Egypt.

The reverse is also true: for some substances, such as certain per- and polyfluoroalkyl substances (PFAS), EFSA has set very low guidance values (2020: 4.4 nanograms per kilogram of body weight per week for the sum of four PFAS). Here, a normal diet can bring some people into the range of the guidance value. That is why we examine each case individually.

Animal and cell studies

Experiments on cells or animals show whether a mechanism is possible. However, doses there are often much higher than in the diet, and animals process substances differently from humans. We always label such studies explicitly. Biological plausibility is not clinical proof.

How our assessments are made

Each claim receives two separate ratings:

1. Verdict on a seven-point scale from “True” to “False”, including “Unclear / insufficient evidence”. A lack of evidence does not mean that a claim is false.
2. Certainty of evidence (high, moderate, low, very low), based on the internationally used GRADE system. It tells how confident we can be about the overall evidence. Observational studies start lower than randomised trials and are downgraded further for shortcomings.

We break broad claims down into sub-claims. “Eggs harm everyone” can be refuted, while “certain people should limit their egg intake” can be correct. We also rate individual studies along several dimensions, such as methodological quality, risk of bias, precision and regional transferability, and we check whether they were reported correctly.

We use tools such as RoB 2, ROBINS-E, AMSTAR 2 and GRADE for orientation. A complete formal assessment with these tools requires full texts, study protocols and usually several independent reviewers. That was not possible here. Our judgements are therefore explicitly labelled as simplified methodological assessments.