Perceived Crisis and Reforms: Issues, Explanations, and Remedies

An overview and discussion is made of the perceived replication crisis in terms of end problems, explanations, and remedies. Following a discussion of a published large-scale replication study and a review of the meta-analysis literature, we come to the conclusion that many effects are small and that on average the variance of the effects is roughly 10% to 25% of the population variance, most likely because of context dependencies. The proposed explanations such as questionable and suboptimal research practices (low power, poor measurement quality, and the use of NHST) are discussed, as well as more distal explanatory factors such as journals, funding agencies, and institutions. Special attention is given to a third kind of explanatory factor that is situated in a psychological reality that comprises small and varying effect sizes. These factors may cause problems even in the absence of researcher-controlled factors. The proposed remedies for proximal problems insist on more power, more replications, more meta-analysis, higher reliability coefficients, and alternatives for NHST. Based on a discussion we formulate critical and less critical remarks regarding these recommendations. We believe there should be room for a diversity of perspectives and approaches. Small and varying effects attributable to complexity and context dependency of human behavior are considered to be great challenges and may lead to an adjustment of research and analysis methods. Finally, open questions are formulated regarding psychological data and data analysis, followed by seven conditional recommendations. (PsycINFO Database Record

Publication year:2018