Elevating Multithreading Further into the Cloud

Due to the surging popularity of cloud computing on one hand and the emergence of numerous novel, innately parallel workloads on the other, executing multithreaded applications in a virtualized setting has become common practice in industry. However, multithreading is known to be highly susceptible to severe performance degradation in virtualized environments. In response, virtualization technologies have evolved rapidly over the years; to the point of virtual machines being considered comparable to their physical counterparts in terms of performance. Precisely because of the rapidity of this evolution however, empirical evidence supporting this consensus is limited at best. Moreover, the crippling levels of performance degradation described in literature less than a decade old suggest that—rapid improvements notwithstanding—it is more than likely that several challenges still remain in this regard. Both identifying and addressing these challenges are the main goals of this work.

Because virtualization is a very broad term, this work commences by describing the virtualization process in general and situating the scope of this Ph. D. project within this broad landscape. Next, it performs a much needed assessment of the state of the art by measuring virtualization overhead for a variety of multithreaded applications through controlled experiments, after first formally defining what exactly virtualization overhead entails within this context. A reflection on potential mitigation techniques for the remaining challenges these experiments lay bare follows. Finally, it refines, implements and evaluates three of the most promising of these techniques, carefully selected to each target a distinct level of the system stack so that they are complementary to one another. 

This Ph. D. makes clear that virtualization overhead is a multifaceted phenomenon, in essence exclusively internal to the system in the form of reduced resource efficiency. Nevertheless, this reduction in resource efficiency may be observable externally in the form of a reduction in temporal efficiency. In particular for multithreaded applications, these system and application effects may differ signifficantly in magnitude. Specifically, this work shows that these effects may still amount to respectively 170% and 80% for multithreaded applications in a state-of-the-art virtualized environment. Although these numbers suggest that much work remains to be done, the complementary mitigation techniques this work elaborates on represent a solid step in the right direction. In particular, chapter 6 presents ’virtual scheduler ticks’ as a means to address excessive virtualization overhead caused by rapid switches between idle and active vCPU states in tickless systems by paravirtualizing the scheduler tick, improving performance by up to 15%. Furthermore, chapter 7 addresses TLB shootdown overhead induced by rapidly resizing application memory space, resulting in the concept of ’global hysteresis’, which yields performance gains of up to 45%. Finally, chapter 8 outlines a series of guidelines application developers may follow to minimize the likelihood of their code suffering significant virtualization overhead. Although the effect of applying these guidelines depends greatly on the nature of the application, the proof of concept included in this manuscript achieves performance improvements of up to 40%.