Frequently Asked Questions

The Problem
A p-value of 0.04 and a p-value of 0.01 are both reported as "statistically significant" — but they can represent vastly different levels of evidence quality that p-values alone do not reveal.

The Solution
Complete statistical evidence requires three dimensions:
- Significance (p): Compatibility with null hypothesis
- Fragility (fr): Classification stability
- Robustness (nb): Distance from neutrality

What is the p-fr-nb triplet?
Complete statistical evidence requires three numbers:
- p: p-value (significance)
- fr: fragility quotient (stability)
- nb: neutrality boundary metric (robustness)
Reporting only p-values provides partial evidence. Complete statistical evidence requires all three dimensions.

Why not just use p-values?
P-values answer one question: "How compatible are the data with no effect?"
They don't tell you:
- How stable is this classification? (fragility)
- How far from neutrality is the result? (robustness)
These are orthogonal dimensions that p-values do not measure.

What is Pattern (1,1,0)?
Pattern (1,1,0) = Clinically Meaningless:
- 1: p-significant
- 1: fr-fragile (unstable)
- 0: nb-weak (near neutrality)
This pattern occurs in 18% of significant trials (13.4-fold elevation over null expectation). P-value says "significant." Complete evidence says "don't trust this."

How is this different from clincalc.com?
The clincalc.com fragility calculator implements a different calculation for the FI than FragilityMetrics.org. We implement the original Walsh (2014) method.

The original Walsh et al. (2014) definition specifies: toggle outcomes in the arm with fewer events; if events are tied, toggle the smaller arm. This tie-breaking rule ensures the FI reflects the minimal perturbation required to flip significance.

Our testing shows that Clincalc.com appears to do the following: when events are equal between arms, its calculator defaults to toggling the control group regardless of arm size. This can produce higher FI values when the control arm is larger than the experimental arm.

Example: Control (10 events, 22 non-events, 32 total patients) vs Experimental (10 events, four non-events, 14 total patients). Per Walsh (2014): events tied → toggle smaller arm (experimental group). This implementation is utilized on FragilityMetrics.org (result: FI=1). In this case, Clincalc.com defaults to toggling the control arm when events are tied (result: FI=2). These are the results from 12/26/2025 (FragilityMetrics, ClinCalc).

Additional features of FragilityMetrics.org:
- We report MFQ (Modified Fragility Quotient), which normalizes FI to the toggled arm, enabling valid cross-study comparison regardless of allocation ratio
- We provide the Global Fragility Index (allows both within and cross-arm toggles; GFI = minimum number of toggles to flip significance) and Global Fragility Quotient (GFQ = GFI/N). See documentation.
- We provide the complete p-fr-nb triplet, not fragility alone
- We report NDI (Neutrality Distance Index) and RQ (Risk Quotient), robustness metrics that clincalc.com does not calculate
- We automatically calculate the "reverse" FI, i.e. if the baseline p-value is non-significant, the FI is the number of toggles required to cross the p=0.05 threshold to become significant.
- Open source code available for verification
- We encourage testing of our software and have made it open source (CC-BY 4.0). Please report any bugs on the GitHub discussion board.

Is this calculator free?
Yes, completely free. No registration required. Open source implementation.

What outcome types are supported?
- Currently supported: Binary (2×2): MFQ + NDI + RQ (this calculator)
- See the Fragility Metrics Toolkit for additional Python calculators covering continuous, diagnostic, ordinal, survival, and correlation outcomes.