Eduovisual
Biostatistics & Population Health
Funnel plot and publication bias detection
— Positive, large-effect, statistically significant studies are preferentially published
— Negative or null trials are delayed, buried in file drawers, or relegated to low-impact journals
— Result: meta-analyses overestimate true effect sizes, sometimes dramatically
— Classic example: SSRI efficacy meta-analyses shrank substantially once FDA registry data (including unpublished negatives) were included
— Reboxetine, oseltamivir, and gabapentin off-label trials all showed major effect-size deflation after unpublished data emerged
— Small-study effects: smaller trials show systematically larger effects than larger trials
— Asymmetric funnel plot
— Heterogeneity (I² elevated) that correlates with study size
— Few trials, industry sponsorship, no trial registration, no protocol pre-registration
— Discrepancy between registry (ClinicalTrials.gov) outcomes and published outcomes
— Outcome reporting bias — selective reporting of favorable endpoints within a published trial
— Time-lag bias — negative trials published years later
— Language bias — non-English negative trials excluded
— Citation bias — positive trials cited more, found more easily
Board pearl: A funnel plot is a visual screening tool, not a diagnostic test. It raises suspicion for small-study effects; confirming publication bias requires statistical tests (Egger, Begg) plus contextual judgment about the literature base.
— Investigator decision not to write up a null trial ("file drawer")
— Sponsor suppression of unfavorable industry-funded results
— Journal editor preference for novel, positive findings
— Peer reviewer skepticism of null results
— Selective outcome switching between protocol and publication
— No entry in ClinicalTrials.gov or WHO ICTRP before enrollment
— Primary outcome in publication differs from registered primary outcome
— Industry sponsor with contractual control over publication
— Multiple subgroup analyses reported as primary
— Conference abstract with positive result never followed by full publication
— Small trials are most vulnerable — easier to suppress, less institutional accountability
— Surgical and device literature is especially prone (no FDA registry mandate equivalent to drug trials historically)
— Pharma-sponsored trials show ~3–4× higher odds of favorable conclusions vs. independently funded
— Pediatric and rare disease literature: few trials, each one disproportionately influential
— Turner et al. NEJM 2008: 31% of FDA-registered antidepressant trials with negative FDA-rated outcomes were unpublished; effect size inflated by ~32% in published literature
— Tamiflu/oseltamivir: Cochrane could not access full clinical study reports for years, distorting influenza guidance
— Vioxx (rofecoxib): selective publication delayed recognition of cardiovascular harm
Key distinction: Publication bias is whether a study appears at all; reporting bias is what gets reported within a published study. Both inflate meta-analytic effects, but funnel plots primarily target the former. Step 3 stems often conflate them — read carefully whether the question is about missing studies vs. missing outcomes.
— X-axis (horizontal): the effect estimate of each study (e.g., log odds ratio, log risk ratio, standardized mean difference, mean difference)
— Y-axis (vertical): a measure of study precision — typically the standard error, plotted inverted (SE = 0 at top, increasing downward)
— Alternative Y-axis: sample size, variance, or 1/SE — but SE inverted is the standard
— Symmetric inverted funnel (wide at the bottom, narrow at the top)
— Large, precise studies cluster near the top around the pooled effect
— Small, imprecise studies scatter symmetrically at the bottom — some overestimate, some underestimate
— 95% pseudo-confidence boundaries drawn as diagonal lines forming the funnel walls; ~95% of studies should fall within
— Bottom-right (or bottom-left, depending on which side favors the intervention) is empty
— Small negative trials are "missing"
— Visual lopsidedness — the funnel looks like a wedge, not a cone
— Think of large trials as the "stable, central" data points — they anchor the truth
— Small trials are the "variable, noisy" periphery — symmetric noise is fine; asymmetric noise = bias
— Is the apex (precise studies) aligned with the pooled estimate?
— Are small studies symmetrically distributed?
— Is there a void on the side of "no effect" or "harm"?
— Are there any extreme outliers driving the pooled estimate?
Board pearl: A funnel plot with <10 studies is uninterpretable — Cochrane explicitly recommends against funnel plot assessment when fewer than 10 trials are pooled, because random scatter mimics asymmetry. This is a classic distractor on Step 3.
— Subjective but the universal starting point
— Look for asymmetry, gaps, and whether large studies cluster near the pooled estimate
— Two reviewers should independently assess; inter-rater agreement is modest at best
— Egger's regression test: regresses the standardized effect (effect/SE) on precision (1/SE); a significantly non-zero intercept suggests asymmetry. Most widely used for continuous outcomes.
— Begg's rank correlation test: correlates standardized effect with variance; less powerful than Egger but distribution-free
— Harbord test: modified Egger for binary outcomes (corrects for the mathematical association between log OR and its SE in sparse data)
— Peters test: alternative for binary outcomes using sample size
— p < 0.10 (note: liberal threshold, not 0.05) typically flags asymmetry
— Low power: requires ≥10 studies, ideally ≥20
— A non-significant test does NOT rule out publication bias — absence of evidence ≠ evidence of absence
— Trim-and-fill (Duval & Tweedie): imputes hypothetical missing studies to make the funnel symmetric, then recalculates the pooled effect. Useful for estimating bias-adjusted effect size but not a correction — it assumes asymmetry = publication bias
— Fail-safe N (Rosenthal): number of null studies needed to nullify the result; criticized as misleading
— PET-PEESE regression: estimates effect at SE=0 (infinite precision)
Step 3 management: When a board stem shows an asymmetric funnel plot, your first move is not to trust the pooled effect at face value — request access to trial registries, look for unpublished data, and interpret the meta-analysis as a likely overestimate.
— Trial registry comparison: cross-check ClinicalTrials.gov, EudraCT, WHO ICTRP against published literature. Unmatched registered trials = candidates for the "file drawer"
— FDA / EMA review documents: for drug trials, regulatory submissions include all trials, published or not. The Turner NEJM 2008 study is the methodological gold standard
— Individual patient data (IPD) meta-analysis: the most rigorous approach; requires raw data from each trial, eliminates many reporting biases
— Cochrane risk-of-bias 2 (RoB 2) tool: domain 5 specifically addresses selective reporting of results
— Overlays contours of statistical significance (p=0.05, p=0.01) on the funnel plot
— If missing studies fall in non-significant zones → suggests publication bias
— If missing studies fall in significant zones of harm → suggests true heterogeneity, not bias
— Helps distinguish publication bias from genuine small-study heterogeneity
— Plot pooled estimate as each new study (in order of size) is added
— Drift toward the null as larger studies enter = classic small-study effect signature
— Narrative assessment
— Comparison of effect sizes between published and registered-but-unpublished trials
— GRADE downgrading for "publication bias" domain based on these qualitative signals
— True heterogeneity (different populations, doses, follow-up)
— Methodological quality differences (small trials often lower quality, inflating effects)
— Chance, especially with few studies
— Choice of effect measure (log OR vs. RR can artificially induce asymmetry)
Key distinction: Asymmetry ≠ publication bias. Asymmetry = small-study effects, of which publication bias is one (common) cause among several. Always consider heterogeneity and quality before concluding bias.
— One of five domains for downgrading certainty of evidence (alongside risk of bias, inconsistency, indirectness, imprecision)
— Downgrade by 1 level for "suspected" publication bias
— Downgrade by 2 levels rarely, for "strongly suspected" with corroborating evidence
— Asymmetric funnel plot with ≥10 studies
— Significant Egger or Harbord test
— Industry-only sponsorship across the evidence base
— Discrepancy between trial registry outcomes and published outcomes
— Few small positive trials, no large confirmatory trial
— Plausible heterogeneity explanation (e.g., dose differences)
— Methodological quality fully explains effect-size differences by study size
— Asymmetry driven by a single outlier
— Low concern: registered protocol, published regardless of result, large confirmatory trial dominates pooled estimate, symmetric funnel
— Moderate concern: mostly small trials, mixed sponsorship, mild asymmetry, no obvious heterogeneity
— High concern: all small trials, all industry-sponsored, marked asymmetry, no registry trail, only positive results in literature, trim-and-fill shifts effect substantially toward null
— High publication bias concern → guideline panels weaken recommendations from "strong" to "conditional"
— May trigger calls for additional pragmatic, independently funded trials before practice change
Board pearl: On Step 3, if a stem describes a meta-analysis of antidepressants/anxiolytics/supplements with effect size that "diminishes substantially when unpublished FDA data are included," the answer is publication bias inflated the original estimate — and the clinical implication is less confident benefit than literature suggests.
— ICMJE policy (2005): journals require registration before first patient enrolled as condition of publication
— ClinicalTrials.gov (US), EudraCT (EU), ISRCTN, WHO ICTRP
— FDAAA 2007: mandates registration and results reporting for most US trials within 12 months of completion — though compliance remains imperfect (~60-70%)
— FDAAA Final Rule (2017): expanded results reporting requirements; civil penalties up to ~$13,000/day for non-compliance (rarely enforced historically)
— EU Clinical Trial Regulation: similar mandate
— AllTrials initiative: advocacy push for retrospective reporting of all past trials
— Statistical analysis plan locked before unblinding
— Reduces outcome switching and p-hacking
— Registered Reports (in some journals): peer review of methods before results known; acceptance based on rigor, not findings
— Vivli, Yale Open Data Access (YODA), ClinicalStudyDataRequest.com
— Enables independent re-analysis
— Results-blind peer review
— Explicit policies welcoming null findings
— Journals dedicated to negative results (Journal of Negative Results in Biomedicine, historically)
— Use Cochrane reviews (rigorous, search unpublished sources)
— Check PROSPERO for the systematic review's pre-registered protocol
— Look for funnel plots and Egger tests in the methods
Step 3 management: When advising trainees about appraising literature, your first-line "drug" is to verify that the systematic review searched trial registries and conference abstracts, not just MEDLINE/Embase. A review that searched only published literature has a built-in publication bias problem regardless of its funnel plot.
— Iteratively removes asymmetric outlying studies, recalculates the center
— Imputes mirror-image "missing" studies on the deficient side
— Recomputes pooled effect including imputed studies
— Limitations: assumes asymmetry is solely from publication bias; can perform poorly with heterogeneity; not a true bias correction — more a sensitivity analysis
— Common board scenario: original OR 0.60, trim-and-fill adjusted OR 0.85 — interpret as "true effect likely smaller than reported"
— Meta-regression of effect size on SE (PET) or variance (PEESE)
— Estimates effect at infinite precision (SE → 0)
— Used heavily in psychology/social science meta-research
— Explicitly model the probability that a study with a given p-value is published
— Copas selection model: parametric, requires assumptions about selection mechanism
— Provides bias-adjusted estimates under specified selection scenarios
— Best used for sensitivity analysis across a range of plausible selection assumptions
— Examine distribution of significant p-values across studies
— Right-skewed (more p<0.01 than p just under 0.05) → genuine effect
— Left-skewed (clustering just below 0.05) → p-hacking / selective reporting
— Useful adjunct, especially for outcome reporting bias
— Incorporate prior beliefs about selection mechanism
— Robust Bayesian meta-analysis (RoBMA) — increasingly cited
— Plots effect vs. SE after accounting for the comparator
— Asymmetry interpreted in network context
CCS pearl: Treat statistical "corrections" for publication bias as sensitivity analyses, not definitive fixes. The honest interpretation: "If publication bias is present, the true effect is plausibly in the range X to Y." Never present a trim-and-fill estimate as the corrected truth.
— Funnel plots and Egger tests unreliable — Cochrane Handbook explicit
— Random scatter creates pseudo-asymmetry
— Use qualitative assessment: registry search, sponsor diversity, time-lag check
— GRADE may still downgrade based on indirect evidence
— Log OR and its SE are mathematically correlated when events are sparse (creates spurious asymmetry on standard funnel plots)
— Use Harbord or Peters tests instead of Egger
— Consider arcsine-transformed proportions
— Standard funnel plots don't work — multiple comparators
— Use comparison-adjusted funnel plots (ordered by comparator hierarchy)
— Asymmetry suggests bias favoring newer/sponsor interventions over established comparators
— Less vulnerable to outcome reporting bias (raw data available)
— Still vulnerable to whole-study publication bias
— Funnel plot still appropriate
— Standard funnel plots inappropriate — sensitivity/specificity are paired, not single effects
— Deeks' funnel plot with effective sample size on Y-axis is the validated alternative
— Asymmetry suggests bias in diagnostic test studies (often sponsor-driven)
— Heterogeneity dominates; funnel plot interpretation extremely limited
— Publication bias likely but hard to disentangle from quality differences
Key distinction: The "special population" in publication bias detection is the evidence base itself — small N of studies, rare events, or unusual designs (network, diagnostic, IPD) each require different funnel plot variants and tests. Misapplying standard Egger to a 6-study binary-outcome meta-analysis is a common Step 3 trap.
— Systematic reviews consistently show industry-funded trials report favorable outcomes ~3–4× more often than independently funded
— Mechanisms: selective publication, favorable comparators (low-dose competitor), surrogate endpoints, post-hoc subgroup analyses
— Funnel plots of industry-only meta-analyses often grossly asymmetric
— FDA Drug Approval Packages (Drugs@FDA): contain reviews of all submitted trials
— EMA European Public Assessment Reports (EPARs): similar transparency
— Clinical Study Reports (CSRs): full detail; increasingly available via EMA Policy 0070 and individual pharma sharing platforms
— Best Pharmaceuticals for Children Act (BPCA) and Pediatric Research Equity Act (PREA): incentivize pediatric studies, with results required to be made public
— Pediatric literature historically prone to publication bias due to small trials
— Historically less regulated than drugs; many devices approved via 510(k) without RCTs
— Surgical innovations rarely have null trials published — strong publication bias
— Boards may highlight robotic surgery, novel implants, or biologics where evidence base is small and sponsor-dominated
— Ghostwriting, contractual restrictions on publication, delayed release of negative data
— Vioxx, Avandia, Paxil pediatric trials — recurrent board examples
— ICMJE form requirement
— Disclosure ≠ mitigation; doesn't fix bias, just transparent about it
Board pearl: When a Step 3 stem describes a meta-analysis where "all included trials were sponsored by the manufacturer" and shows an asymmetric funnel plot, the correct interpretation is publication bias is likely, the pooled effect probably overestimates true benefit, and the appropriate recommendation is to seek independently funded confirmatory data before changing practice.
— Overestimation of efficacy: patients exposed to therapies less effective than literature suggests
— Underestimation of harm: safety signals suppressed (e.g., rosiglitazone CV risk, Vioxx MI, SSRI pediatric suicidality)
— Misdirected resources: costly interventions adopted based on inflated benefit
— Delayed adoption of effective but "unexciting" therapies whose modest effects look unimpressive next to inflated competitors
— Class I recommendations based on biased evidence → widespread practice change
— Reversal of recommendations (e.g., HRT for primary prevention, intensive glucose control in T2DM) often follows recognition of biased evidence base
— "Medical reversal" literature — Prasad/Cifu: ~40% of standard practices later found ineffective or harmful
— Inflated cost-effectiveness ratios (numerator: benefit overstated)
— Formulary decisions based on biased meta-analyses
— Insurance coverage of marginal interventions
— Wasted resources duplicating "known" findings that are actually null
— Ethically problematic — patients enrolled in trials whose results may be suppressed
— Erosion of public trust when reversal occurs
— Vioxx withdrawal, settlements, FDA reforms (FDAAA 2007 directly traced to recognition of suppressed CV data)
— Paroxetine Study 329 — published as favorable for adolescents; reanalysis from raw data showed no efficacy and increased suicidality
— Trust in clinician → trust in evidence → trust in guideline → exposure to inflated benefit
— Each link multiplies harm when underlying evidence is biased
Step 3 management: When counseling a patient about a new therapy backed by a small evidence base, explicitly acknowledge uncertainty ("the published evidence may overstate benefit") rather than overstating. Shared decision-making requires honest communication of evidentiary limits, especially for marketed treatments backed primarily by industry-sponsored trials.
— Strong clinical implications (mortality, major morbidity) hinging on a biased-appearing evidence base
— Discrepancy between RCT meta-analysis and pragmatic / real-world data
— Marked asymmetry confirmed by multiple tests
— Few large independent trials; many small industry-sponsored trials
— Trim-and-fill substantially shifts pooled estimate
— Step 1: Search PROSPERO and trial registries for unpublished or in-progress trials
— Step 2: Request Clinical Study Reports via EMA, FDA FOIA, or pharma data-sharing portals (Vivli, YODA)
— Step 3: Conduct or commission individual patient data (IPD) meta-analysis
— Step 4: Commission a pragmatic, publicly funded confirmatory trial (NIH, PCORI, Cochrane-affiliated)
— Biostatistician for advanced bias-adjustment methods
— Information specialist / medical librarian for comprehensive registry and gray literature searches
— Methodologist trained in GRADE for evidence certainty assessment
— Clinical content expert for distinguishing heterogeneity from bias
— Guideline panels: include methodologists and patient representatives, mandate conflict-of-interest management
— Hospital P&T committees: weight pooled estimates by evidence certainty, not raw effect size
— Payers: increasingly demand independent HTA review (e.g., ICER in US, NICE in UK)
— Cardiovascular outcome trials (FDA mandate for diabetes drugs post-rosiglitazone)
— Pragmatic effectiveness trials after efficacy signals
— Long-term safety surveillance via registries
CCS pearl: In a "real-world" CCS-style stem, if you're asked what next step a hospital committee should take regarding a meta-analysis with suspected publication bias, the right answer is rarely "trust the pooled estimate" — it's "defer adoption pending independent confirmatory evidence" or "adopt with explicit uncertainty acknowledgment and outcome monitoring."
— Small trials may study different populations, doses, durations
— Example: smaller trials enroll sicker patients with more potential to benefit → larger effects, not bias
— Differentiator: asymmetry persists in subgroup-stratified plots; meta-regression reveals genuine moderators
— Smaller trials often have less rigorous design (inadequate allocation concealment, no blinding, high attrition)
— Poor quality systematically inflates effects → spurious asymmetry
— Differentiator: sensitivity analysis restricted to high-quality trials; asymmetry attenuates
— With <10 studies, scatter mimics asymmetry
— Differentiator: none reliable; just acknowledge limitation
— Log OR vs. RR vs. RD can artificially induce or hide asymmetry, especially with extreme baseline risks
— Differentiator: repeat funnel plot with alternative effect measure
— For binary outcomes with sparse events, log OR and its SE are mathematically linked
— Standard Egger test inflates type I error
— Differentiator: use Harbord, Peters, or arcsine-transformed analyses
— Contour-enhanced funnel plot: if missing studies fall in significant zones → bias; if in non-significant zones → could be heterogeneity
— Subgroup analyses by quality, population, dose
— Meta-regression of effect on plausible moderators
— Sensitivity analysis excluding small/low-quality trials
Key distinction: The funnel plot is a screening test with multiple possible "diagnoses" for asymmetry. Publication bias is the most concerning but not the only, and not always the most likely. Step 3 stems may set a trap by presenting an asymmetric funnel with substantial clinical heterogeneity — the "best answer" may be heterogeneity, not bias.
— Trial published, but only favorable outcomes reported
— Primary endpoint switched from protocol to publication
— Detected by: protocol vs. publication comparison, ORBIT classification system
— Funnel plot does not detect this directly
— Positive trials published faster than negative ones (median delay ~1–3 years longer for negatives)
— Affects early meta-analyses of emerging therapies disproportionately
— Detected by: cumulative meta-analysis ordered by publication date
— Negative findings preferentially published in non-English journals (especially for non-Anglophone investigators with English as second language)
— Reviewers restricting to English literature may inflate effect
— Mitigated by: multilingual search strategies
— Positive trials cited more, easier to find via snowballing
— Mitigated by: systematic database searching, not citation tracking alone
— Same trial published in multiple papers; meta-analysis double-counts if not detected
— Mitigated by: careful deduplication; check author lists and registration numbers
— Indexing biases — some journals not in MEDLINE
— Mitigated by: searching Embase, CINAHL, regional databases, gray literature
— Conference abstracts, theses, government reports — often contain negative findings
— Mitigated by: searching ProQuest dissertations, OpenGrey, conference proceedings
Board pearl: A complete "publication bias assessment" in a Cochrane-quality review addresses multiple bias types, not just the funnel plot. If a board stem asks about the single best method to detect publication bias, the answer is usually "comparison of published vs. registered/regulatory data" rather than the funnel plot itself — because the funnel plot detects consequences of bias, while registry comparison detects the missing studies directly.
— Universal trial registration before enrollment
— Enforcement of results posting (FDAAA penalties, EU CTR mandates)
— Linkage of registry records to published papers
— PROSPERO registration before data extraction
— Locks in protocol, prevents post-hoc tweaking of inclusion criteria
— Open data, open code, open peer review
— Registered Reports format: methods accepted before results known
— Preprint servers (medRxiv, bioRxiv) reduce time-to-public-availability of all findings
— Commitment to publish methodologically sound studies regardless of result
— Welcoming replication studies and null findings
— ICMJE requirement: clinical trials must be registered to be considered for publication
— NIH Public Access Policy: funded research must be deposited in PubMed Central
— Wellcome Trust, EU Horizon: similar open-access mandates
— Some funders now require results posting as condition of future grants
— Routinely check Cochrane reviews for funnel plot and Egger test in methods
— Use GRADE-rated guidelines preferentially
— Be skeptical of meta-analyses with all small, all industry-funded trials
— Update knowledge as larger pragmatic trials emerge — "guideline humility"
— Modern UpToDate / DynaMed flag certainty-of-evidence per recommendation
— Use these certainty ratings, not just direction of recommendation, in counseling
Step 3 management: As a practicing physician, your "secondary prevention" toolkit includes (1) preferring guidelines that use GRADE, (2) trusting Cochrane reviews over single industry-sponsored meta-analyses, (3) re-evaluating practices when large independent trials publish (especially CV outcome trials and pragmatic effectiveness trials), and (4) discussing uncertainty with patients during shared decision-making.
— Continuously updated reviews (Cochrane Living Reviews)
— Re-run searches and re-assess funnel plots as new trials publish
— Particularly important in rapidly evolving fields (COVID-19 therapies were a landmark example)
— Plot pooled estimate over time as each study is added
— Detect when evidence first crossed a clinically meaningful threshold
— Reveal time-lag bias (estimate "drifting" toward null as time progresses suggests early positive trials dominated initially)
— Required for many newly approved drugs
— REMS programs (Risk Evaluation and Mitigation Strategies) — FDA-mandated risk monitoring
— Pharmacovigilance signals (FAERS, EudraVigilance, WHO VigiBase)
— Real-world evidence increasingly supplements RCT-based estimates
— Disease and device registries (e.g., STS Cardiac Surgery Database, NCDR, ACR RISE)
— Capture outcomes regardless of publication, mitigating bias in surgical/device literature
— Major guidelines (AHA/ACC, ADA, GOLD, USPSTF) update on 3–5 year cycles, sometimes interim
— Reflect emerging evidence and corrections to prior overestimates
— Clinicians should periodically audit personal prescribing against current high-certainty evidence
— Be willing to de-implement practices shown ineffective ("low-value care" initiatives — Choosing Wisely)
— Discuss evolving evidence with long-term patients (e.g., "we used to recommend X; new data suggest otherwise")
— MOC/CME requirements emphasize evidence-based practice
— Journal clubs and academic detailing for continuing critical appraisal skills
CCS pearl: Long-term "monitoring" for publication bias in your own practice = staying current with updated guidelines, watching for medical reversals, and being willing to change practice when high-quality pragmatic trials contradict earlier biased meta-analyses. Inertia is a form of clinical harm.
— Declaration of Helsinki (2013 revision, ¶36): "Researchers have a duty to make publicly available the results of their research on human subjects... Negative and inconclusive as well as positive results must be published or otherwise made publicly available."
— Failure to publish = breach of trust with trial participants who consented to advance knowledge
— IRBs increasingly require publication plans as part of approval
— FDAAA 2007 (US): mandatory registration and results reporting; penalties up to ~$13,000/day, recently increasing in enforcement
— EU Clinical Trial Regulation 536/2014: results within 12 months (6 for pediatric)
— EMA Policy 0070: proactive publication of clinical reports submitted with marketing applications
— Patients consenting to trials are told their participation will contribute to medical knowledge — selective non-publication breaks that promise
— At the bedside, consent for therapies should reflect honest representation of evidence base, including uncertainty from suspected publication bias
— Discharge regimens based on inflated evidence may underperform in real-world follow-up
— When new evidence emerges showing prior practice was based on biased data, communicate proactively with affected patients (e.g., HRT for primary prevention reversal)
— Document shared decision-making conversations including evidentiary uncertainty
— Suppressed safety data = patient harm (rofecoxib, rosiglitazone)
— FDA MedWatch and institutional adverse event reporting are downstream safeguards but rely on signal detection from imperfect literature
— ICMJE COI disclosure mandatory
— Authors with industry COI not disqualified but must disclose; readers and editors must weight accordingly
— Guideline panel composition: many bodies (NICE, USPSTF) now limit voting members with industry COI
— False Claims Act has been used against pharma suppression of trial data (qui tam suits)
— Researchers fearing reprisal for publishing negative findings have legal protections under federal whistleblower statutes
Board pearl: A Step 3 ethics stem may describe an investigator pressured by a sponsor not to publish a null finding — the correct action is publish anyway, citing the Declaration of Helsinki and the ethical obligation to research participants. Sponsor contractual control over publication is increasingly considered unethical and is rejected by reputable academic centers.
— Funnel plot interpretable with ≥10 studies (Cochrane Handbook)
— Egger test typical threshold: p < 0.10 (not 0.05) due to low power
— Antidepressant publication bias study (Turner NEJM 2008): 31% of FDA-registered trials unpublished, effect size inflated ~32%
— Industry-sponsored trials: ~3–4× more likely to report favorable conclusions
— Trim-and-fill: developed by Duval & Tweedie (2000)
— Egger: linear regression test for funnel asymmetry (1997)
— Begg: rank correlation test (1994)
— Harbord: modified Egger for binary outcomes
— Peters: alternative test for binary outcomes
— Deeks: funnel plot for diagnostic test accuracy reviews
— Duval & Tweedie: trim-and-fill
— Copas: selection model
— Rosenthal: fail-safe N
— Standard funnel: SE on Y-axis (inverted), effect on X
— Contour-enhanced funnel: significance contours overlaid
— Deeks' funnel: for diagnostic accuracy
— Comparison-adjusted funnel: for network meta-analyses
— Antidepressants (Turner 2008)
— Oseltamivir (Tamiflu) — Cochrane re-review with CSRs
— Reboxetine — efficacy collapsed with unpublished data
— Rofecoxib (Vioxx) — suppressed CV harm signals
— Paroxetine Study 329 — adolescent depression, RIAT reanalysis
— Rosiglitazone (Avandia) — suppressed CV risk → FDA reform
— ICMJE registration mandate: 2005
— FDAAA: 2007 (results reporting mandate)
— EU CTR: 2014 (effective 2022)
— EMA Policy 0070: 2014
Key distinction: Don't confuse publication bias (study-level missingness) with selective outcome reporting (within-study missingness) — both inflate meta-analytic effects, but only the former is what funnel plots target.
— Stem: meta-analysis of [intervention]; funnel plot shows small studies clustered to one side
— Best answer: publication bias likely; pooled effect probably overestimates true effect
— Distractors: heterogeneity (consider but usually not "best" if asymmetry is one-sided toward favorable), random chance, methodological quality
— Stem: "When FDA registry data including unpublished trials are added, the effect size diminishes from OR 0.50 to OR 0.75"
— Best answer: publication bias inflated the published estimate
— Implication: clinical recommendation strength should be reduced
— Stem: 6-study meta-analysis with apparent funnel asymmetry
— Best answer: funnel plot interpretation unreliable with <10 studies; cannot reliably assess publication bias visually
— Distractors that look tempting: "publication bias present" (wrong — insufficient data)
— Stem: original pooled OR 0.60; after trim-and-fill, OR 0.85
— Best answer: publication bias likely inflated original effect; adjusted estimate suggests smaller true effect
— Don't choose: "the corrected estimate proves the true effect is 0.85" — trim-and-fill is sensitivity, not correction
— Stem: all 8 included trials manufacturer-funded; asymmetric funnel
— Best answer: publication bias likely; recommend independent confirmatory trials before practice change
— Stem: sponsor asks investigator to withhold a null result
— Best answer: publish the trial; participants consented based on contribution to knowledge (Declaration of Helsinki)
— Stem: meta-analysis of binary outcomes with sparse events
— Best answer: use Harbord or Peters test instead of Egger for funnel asymmetry
— Wrong: Egger's linear regression alone
— Stem: meta-analysis of sensitivity/specificity
— Best answer: use Deeks' funnel plot, not standard funnel plot
Step 3 management: When a stem combines a funnel plot image with a clinical management decision, the highest-yield answer integrates methodological skepticism with patient-centered counseling — neither blind trust in the meta-analysis nor reflexive rejection, but explicit acknowledgment of evidentiary uncertainty in shared decision-making.
A funnel plot is a visual screening tool for publication bias in meta-analyses — plotting each study's effect size against its precision (SE inverted on Y-axis) and looking for asymmetry that suggests small, unfavorable studies are systematically missing from the literature.