This Week We Focus On The Concept Of False Discovery In Data

This week we focus on the concept of false discovery in data

This week we focus on the concept of false discovery in data. After reviewing the article by Naouma (2019), answer the following questions: What is a false discovery rate? Can a false discovery rate be completely avoided? Explain. What was the outcome of the results of the use case?

Read: ch.10 in textbook: Avoiding False Discoveries Naouma, P. (2019). A comparison of random-field-theory and false-discovery-rate inference results in the analysis of registered one-dimensional biomechanical datasets . PeerJ (San Francisco, CA), 7, e8189–e8189.

Paper For Above instruction

False discovery rate (FDR) is a statistical measure used to account for multiple comparisons or hypothesis testing, especially in fields like neuroimaging, genetics, and biomechanics where numerous tests are performed simultaneously. It represents the expected proportion of false positive findings—incorrect rejections of the null hypothesis—among all the declared significant results. Managing FDR is crucial because performing many tests increases the likelihood of identifying false positives, which can lead to incorrect conclusions and misguided subsequent research or interventions.

Naouma's (2019) study investigates the effectiveness of two statistical correction methods—random-field theory (RFT) and the false discovery rate (FDR)—when analyzing biomechanical datasets. The core question addressed is whether FDR can be completely avoided and how this impacts the validity of the findings. While methods like FDR control are designed to limit the proportion of false positives, they inherently cannot eliminate the possibility of false discoveries entirely. This is because statistical procedures, by their nature, balance sensitivity (detecting true effects) against specificity (avoiding false positives). FDR control adjusts for multiple comparisons, but randomness and data variability mean some false discoveries may still occur, particularly in complex datasets with high noise levels or small effect sizes.

In Naouma’s (2019) case study, the application of FDR and RFT yielded differing outcomes but highlighted similar issues. The results demonstrated that FDR tends to be more conservative, reducing false positives but sometimes at the cost of missing genuine effects (Type II errors). Conversely, RFT might detect more significant effects but can be more prone to false positives if assumptions are violated. The study's findings underscore that while FDR significantly reduces the risk of false discoveries, it cannot guarantee their complete absence. Researchers need to consider the context of their data, the consequences of Type I versus Type II errors, and choose the appropriate correction method accordingly.

The outcome of the use case revealed that FDR offers a robust approach for controlling false positives in biomechanical data analysis, especially when multiple tests are involved. Naouma (2019) concluded that combining statistical methods with a clear understanding of their limitations enhances the reliability of research findings. Ensuring reproducibility and accuracy in biomechanical research depends on selecting suitable correction strategies that align with the study objectives and data characteristics, acknowledging that zero false discoveries are mathematically impossible to achieve.

Overall, controlling the false discovery rate is a vital aspect of rigorous scientific analysis. While advances in statistical methodologies improve our ability to limit false positives, complete elimination is unattainable. Continuous methodological improvements, rigorous data quality control, and transparent reporting are essential to mitigate the impact of false discoveries in scientific research.

References

• Naouma, P. (2019). A comparison of random-field-theory and false-discovery-rate inference results in the analysis of registered one-dimensional biomechanical datasets. PeerJ, 7, e8189.
• Benjamini, Y., & Hochberg, Y. (1995). Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the Royal Statistical Society. Series B (Methodological), 57(1), 289–300.
• Genovese, C. R., et al. (2002). Thresholding of Statistical Maps in Functional Neuroimaging Using the False Discovery Rate. NeuroImage, 15(4), 870–878.
• Friston, K. J., et al. (2002). Statistical parametric mapping: The analysis of functional brain images. Academic Press.
• Silva, S., et al. (2020). Comparison between FDR and RFT in neuroimaging studies. Neuroinformatics, 18(2), 251–265.
• Qian, Q., et al. (2019). Multiple testing procedures in neuroimaging: a review. Statistical Methods in Medical Research, 28(3), 646–668.
• Nichols, T. E., et al. (2017). Best practices in data analysis and sharing in neuroimaging using MRI. Nature Neuroscience, 20(3), 356–360.
• Yekutieli, D., & Benjamini, Y. (1999). Resampling-based false discovery rate controlling multiple test procedures for correlated test statistics. Journal of Statistical Planning and Inference, 82(1-2), 171–196.
• Subrach, V., et al. (2018). False discovery rate correction methods in real-world data. Journal of Applied Statistics, 45(4), 766–786.
• Winkler, A. M., et al. (2016). Permutation inference for the general linear model. NeuroImage, 143, 630–647.