Pennies make pounds: Multilevel meta-analysis of single-case experimental data

Single-case experimental designs (SCEDs) have been frequently used in different disciplines to evaluate the effectiveness of interventions or treatments (Schlosser, Lee, & Wendt, 2008; Shadish, 2014a, 2014b; Shadish & Rindskopf, 2007; Smith, 2012). In this kind of design one or a few cases are repeatedly measured under different conditions (Barlow, Nock, & Hersen, 2009; Kratochwill et al., 2010; Onghena, 2005). Although SCEDs are intrinsically used for idiographic research (when the interest is in a specific case rather than the 'mean' across a population), one of the main issues of this kind of designs is their limited generalizability and hence their limited usefulness for the development of evidence-based decisions because of the small number of cases under investigation. To enhance generalizability, researchers replicate SCEDs across cases, across settings, or across behaviors. SCED studies can be combined to enhance generalizability and for that purpose, meta-analytic techniques can be used. Meta-analytic procedures allow researchers to quantitatively synthesize the results of these replications and provide evidence for best practices (Beretvas & Chung, 2008a; Petit-Bois, Baek, Van den Noortgate, Beretvas, & Ferron, 2016; Tincani & De Mers, 2016). The interest in the meta-analysis of SCEDs has increased in the past decades (Shadish, 2014a; Shadish, Hedges, & Pustejovsky, 2014). SCED meta-analysts can apply different methods and procedures for summarizing the data from multiple SCED studies such as the calculation of a simple average, standardized mean difference, median, or range of effect sizes, a regression analysis of effect sizes, or a multilevel analysis of raw data or effect sizes. This dissertation focuses on extending multilevel modeling for synthesizing SCEDs with twofold objectives. The first objective is to provide a comprehensive overview on published and unpublished SCED meta-analyses to provide a better insight about what is going on in SCED meta-analyses and to establish some empirical foundations for further research by SCED investigators, meta-analysts, and methodologists. Secondly, it aims to contribute to handling some complexities when aggregating multiple SCEDs using three-level meta-analytic approach, in particular handling bias in variance component estimation and handling dependencies in the case of multiple regression coefficients. After a general introduction (Chapter 1), the second chapter (Chapter 2) explores the general characteristics of previously conducted SCED meta-analyses, namely design characteristics, the kind of data provided in the primary studies and the ones included in meta-analyses, and the type of analyses that were conducted. Chapter 3 investigates the methodological quality of SCED meta-analyses. Through a comprehensive methodological quality assessment of SCED meta-analyses, this chapter further clarifies important deficiencies in the validity and credibility of the results of these meta-analyses. In Chapter 4, we briefly review some existing tools for assessing the quality of systematic reviews/ meta-analyses that might also be helpful for SCED meta-analyses to organize and evaluate the quality and reliability of the findings. Chapter 5 evaluates the performance of some adjustment approaches in combining standardized SCED data with small measurement occasions (20 or fewer) in multilevel meta-analysis proposed by Van den Noortgate and Onghena (2003a, 2003b, 2008) for estimating fixed effects and variance components in three-level meta-analyses of standardized raw data and standardized effect sizes. In Chapter 6, we try to handle the dependence among multiple regression coefficients representing the treatment effects when meta-analyzing data from single-case experimental studies. Finally, in the last chapter (Chapter 7), a summary of findings is given, following with methodological issues and limitations, implications for SCED researchers, meta-analysts, and for further research.

Publication year:2019

Accessibility:Closed