How good is a study? Evidence and recommendations explained

What “S3” means

According to the 2021 guideline, an S3 guideline is based on a systematic search, selection and assessment of scientific evidence on the relevant questions and is developed according to the criteria of the AWMF. It reflects the state of research at the time. ^{[1, Ch. C.1]} Studies from other health systems are assessed for how well they can be transferred to the German health and social system. ^{[1, Ch. C.1]} A guideline is a snapshot at a particular point in time; that is why expired guidelines should only be cited with a time frame: The S3 guidelines on autism.

“Soll”, “sollte”, “kann”: grades of recommendation

For the evidence-based recommendations, the guideline distinguished according to the AWMF approach ^{[1, Ch. C.1]}:

Wording (German)	Meaning according to the guideline	Grade of recommendation
“soll” (should)	strong recommendation based on the evidence	A
“sollte” (should, weaker)	recommendation based on the evidence	B
“kann” (may)	open recommendation because of low evidence	0

This helps with reading: “kann” is not an instruction but an open possibility. Examples from other articles: Augmentative and alternative communication, Medication in autism.

Consensus and “good clinical practice”

Not every recommendation rests on studies. Where the body of studies is not sufficient, recommendations were formulated and agreed on the basis of the expertise of the members of the guideline group; they are marked “KKP” (“good clinical practice”, clinical consensus point). ^{[1, Ch. C.1]} Each recommendation also states the strength of consensus of the vote, for example “strong consensus (>95 % agreement)”, “consensus (>75 % to 95 %)” or “majority agreement (50 % to 75 %)”. ^{[1, Rec. 31, 109]} A consensus-based recommendation is therefore a joint judgement by experts, not a study result.

Types of study and risk of bias

In its tables, the guideline named, among others, RCT (randomised controlled trial), CCT (clinically controlled trial without randomisation) and the risk of bias (“Risk of Bias”, RoB). It gave the evidence level according to the Oxford Centre for Evidence-Based Medicine (CEBM). ^{[1, Table for Ch. C.4.2]} Editorial explanation (general, not from the sources): in a randomised trial, chance decides who receives a treatment; this makes comparisons more reliable. In observational studies, groups are observed without the researchers assigning them; there, other influences have to be accounted for statistically. A meta-analysis combines the results of several studies; its informative value depends on how good the individual studies are.

Two examples: meta-analysis and cohort study

• Meta-analysis: A 2014 meta-analysis evaluated five cohort studies with 1,256,407 children and five case-control studies with 9,920 children on vaccinations and autism. ^[3] For the cohort studies, it gives, for example, an odds ratio of 0.99 with a 95 % confidence interval of 0.92 to 1.06. ^[3] Editorial explanation: an odds ratio close to 1 means practically no difference between the groups compared; the confidence interval gives the range in which the true value lies with high probability. If it includes 1, a difference is not shown. In detail: Do vaccinations cause autism?.
• Cohort study: A Danish register study followed 657,461 children and found a hazard ratio of 0.93 (95 % confidence interval 0.85 to 1.02) for MMR-vaccinated compared with unvaccinated children. As a limitation, the authors note that no individual patient records were checked. ^[4]

Both examples show: figures alone are not enough. What matters are the study design, the number of participants, how other influences are accounted for and the limitations stated.

Questions you can ask yourself about a news report

Editorial orientation, not a requirement from the sources:

1. Where does the statement come from? Study, guideline, authority, individual case report?
2. How was it investigated? Who was compared, how many people, for how long?
3. What do the authors themselves say about limits? Reputable work names them.
4. Does this apply to you? Age, intelligence, setting and country can differ; the guideline therefore checked transferability. ^{[1, Ch. C.1]}
5. Is more promised than shown? On this: Recognising dubious offers.

What else you should know

• No evidence is not the same as “does not work”. The guideline stressed this explicitly. ^{[1, Ch. C.1]} Equally, “evidence available” does not mean something suits every person.
• Body of studies on autism: The guideline considered the body of studies on autism to be clearly better than for some other mental disorders in childhood and adolescence. ^{[1, Ch. C.1]}
• New studies are put in context by us in News, for example Autism prevalence worldwide.

Sources and literature

1. DGKJP, DGPPN et al. Autismus-Spektrum-Störungen im Kindes-, Jugend- und Erwachsenenalter. Teil 2: Therapie. S3-Leitlinie, AWMF-Reg.-Nr. 028-047, Stand Text 02.05.2021 (Kap. C.1, Tabellen- und Abkürzungserläuterungen zu Kap. C.4.2, Konsensstärken der Empfehlungen). – Guideline (in German); expired on 23 March 2026 (see no. 2)
2. DGKJP. Aktuelle Leitlinien (Eintrag 028-047: „Gültig bis 23.03.2026“). https://www.dgkjp.de/wissen/aktuelle-leitlinien/ – Professional society (website in German)
3. Taylor LE, Swerdfeger AL, Eslick GD. Vaccines are not associated with autism: an evidence-based meta-analysis of case-control and cohort studies. Vaccine 2014;32(29):3623–3629. https://doi.org/10.1016/j.vaccine.2014.04.085 – Meta-analysis
4. Hviid A, Hansen JV, Frisch M, Melbye M. Measles, mumps, rubella vaccination and autism: a nationwide cohort study. Annals of Internal Medicine 2019;170(8):513–520. https://doi.org/10.7326/M18-2101 – Cohort study