DEVELOPMENT OF TOOLS FOR PERSONALIZED MENTAL HEALTH CARE - Clinical use of Experience Sampling Methods

This PhD thesis aimed to address challenges in the development and implementation of clinically-oriented tools for using the Experience Sampling Method (ESM) in routine clinical practice. The ESM is a technique for capturing and quantifying mental states and symptoms as they occur in daily life. It involves prompting individuals to fill out a questionnaire multiple times per day for several days in order to measure present ‘in-the-moment’ experiences. The ESM brings several advantages to the clinic. It allows for accurate reporting of contexts and mental states without needing to heavily rely on retrospective memory. Furthermore, it can be used to study in situ emotion regulation, associations between symptoms of mental health problems, and treatment efficacy. Finally, ESM can additionally be used to help increase self-insight and provide interventions when needed. Hence, ESM offers promising benefits to clinical practice. However, it is not yet routinely used.

Four prominent challenges concerning the implementation of ESM include the need to: (1) map expectations on how practitioners intend to use ESM, (2) evaluate whether current ESM software is fit for clinical use, (3) develop methods for reliable data analysis, and (4) improve pragmatic aspects such as user-friendliness. To tackle these challenges we conducted a series of studies. The first study investigates user requirements and desires for clinical software. The second study evaluates existing features of ESM apps and dashboards. The third study demonstrates the use of multiverse analysis — a statistical technique used to investigate the effect of preprocessing choices on statistical analysis outcomes — for increasing the reliability of ESM data analysis. The fourth study combines findings from the first three studies, as well as existing literature, to pilot the use of an ESM protocol in clinical practice. The protocol provided practitioners with software that automatically enrolls clients into a week of ESM data collection. Additionally, data was visualized and practitioners were given training. This final study was done to help identify implementation challenges related to user experience.

In the first study, addressed in chapter 3, our findings showed that practitioners are enthusiastic about using ESM and foresee multiple practical applications that could, each on their own, form a focus for implementation. However, practitioners found it difficult to interpret and use ESM data visualizations. Several barriers and facilitators to the uptake of ESM were additionally identified, including the facilitator of 'reimbursement for (learning to) use', which suggests that practitioners view ESM as additional labor. Hence, it is necessary to focus on strategies that promote ESM as a tool to improve mental healthcare. 

In the second study, addressed in chapter 4, we found that most ESM apps and dashboards focus primarily on academic use, which may present a problem for practitioners who need data analysis and visualization in real-time. Challenges such as continued software functionality, legal requirements, and financial sustainability were identified. We concluded that the field is lacking ESM apps and dashboards that focus on clinical use.

In the third study, addressed in chapter 5, we demonstrated the use of multiverse analysis to assess the impact of preprocessing choices such as data exclusion based on compliance, exclusion of the first assessment day, and the calculation of constructs as the mean, median, or mode. Our results showed that conclusions were not affected by data exclusion decisions. Group differences in negative affect between individual with psychosis as compared to controls were affected when negative affect was calculated as the mean compared to the median or mode, and this was attributed to differences in the within- and between-factor structure of negative affect. With this study, we illustrated the value of using multiverse analysis to ensure the validity of results and re-opened a discussion for ESM construct validation.

In the fourth study, addressed in chapter 6, we found mixed opinions concerning questionnaire content, personalization, and data visualization of the implemented ESM template. The software and interactive use of data were considered reasonable to good, but practitioners experienced initial difficulties learning to use the software and attributing value to the data visualizations. The training material was considered insufficient and there were considerable inter-individual differences among clients regarding usability ratings and compliance. Lessons learned from this study include the need to offer various sampling schedules tailored to different clinical scenarios, adding an open-text field item, a customizable visualization dashboard, realistic goal setting, and an extensive training program covering practical aspects of ESM.

Considering the findings presented above, this PhD thesis addressed the outlined challenges in the development and implementation of clinically-oriented ESM apps and dashboards. It provided clear indications on how practitioners want to use ESM and elucidated what facilitates or withholds use. An overview of existing ESM apps and dashboards was similarly provided, as well as a method for investigating reliability of data analysis. Next, remaining implementation challenges were identified through piloting an ESM app and dashboard developed with end-user input. Questions remain on implementation approach, format of ESM, and types of technology to conduct ESM. Further research is needed on streamlining ESM to clinical applications, economic evaluation, and increasing user engagement. Advantages to this PhD project include the input from stakeholders with various backgrounds, an implementation approach based on a realistic scenario, and our output demonstrating practical implications for future work. Limitations included being unable to iteratively improve the developed ESM app and dashboard, being confined to the challenges previously decided on, and having — perhaps — a sampling bias in our work. Regardless, this PhD project is believed to facilitate the clinical implementation of ESM apps and dashboards.