Fig. 01 Where the null p-values land
Empirically Calibrated Conditional Independence Tests
Empirically calibrated conditional independence testing (ECCIT) for false discovery rate control.
Summary
Conditional independence tests drive causal discovery and feature selection, but in practice they often break their error guarantees, even with false discovery rate (FDR) control. Small samples undermine asymptotic guarantees, and misspecified models skew p-values under the null. ECCIT measures and corrects this miscalibration. For a base test such as GCM or HRT, an adversary searches for the features and response functions that miscalibrate it most, and ECCIT then fits a monotone map that adjusts the test’s p-values to match. On synthetic and real data, it achieves valid FDR with higher power than existing calibration methods.
Method
-
01
Fit a worst-case adversary
Search a class of plausible responses for the one that breaks the test worst: which features matter (γ) and how (f).
-
02
Generate adversarial data
Resample rows of the real X and draw outcomes Ỹ from that adversary, so we know exactly which features are null.
-
03
Measure the miscalibration
Run the base test (GCM, HRT, …) on each resample. Its null p-values should be flat; the pile-up near zero is the error.
-
04
Fit the calibrator
Learn a monotone map from raw to calibrated p-values, then apply it to the test’s p-values on the real data.
Calibration
False discoveries
Published in
Proceedings of the 29th International Conference on Artificial Intelligence and Statistics (AISTATS), PMLR 300:5194–5202, 2026
