Audio classification with resource-constrained wireless sensor networks

Smart sensing has been subject of significant academic and industrial interest, as these services are becoming more popular due to integrated devices shrinking in size while maintaining their computational power. When designing an integrated solution for a given application, an important choice is to select a proper set of sensor modalities along with the amount of sensors that are needed to cover the environment. In this dissertation, applications of interest are those that benefit from a network of sensors, and specifically that of microphones. Such a sensor is attractive as it can convey highly informative data while keeping a moderate energy consumption. In order to allow ubiquitous sensing, a wireless acoustic sensor network (WASN) is considered. In many applications battery-fed integrated devices or sensor nodes are preferred since they relax the installation efforts. However, this raises additional constraints on the energy budget to enable an appropriate autonomous lifetime. This thesis investigates methods to reduce the energy consumption of a system that automatically classifies audio waveforms that are collected by the WASN. Although the main contributions of this work are generally applicable, all solutions were primarily evaluated on the use case of classifying domestic activities in a home environment. In order to have a detailed view on the energy consumption of the main components of a WASN, a multi-layered energy behavioural model was proposed. The model is multi-layered as the energy consumption of the sensor-, processing-, and communication layer is modelled separately. Given this model, it was motivated to opt for a paradigm where model inference is performed at the battery-fed sensing node such that only classification outputs are transmitted to a central level. In this way the energy consumption at the communication layer is reduced while that of the processing layer is increased but with a lesser amount. To further lower the energy consumption of the local processing layer two methods are proposed in this thesis. A first method leverages the redundancy of information that is available over different sensing nodes. As different sensor nodes in a network might monitor the same sound sources but from a different point of view, redundancy may exist. As a consequence the data from only a subset of the sensing nodes can be used. Three different dynamic sensor (de-)activation algorithms were proposed that only use the classification's output to decide which sensor nodes need to be active and share information at the central level. It was indicated that the algorithm employing central control reduced the average energy consumption per sensor node by 80%. A second method reduces the energy cost related to perform model inference at the sensor node. It is a well-known fact that current neural networks are overparameterised, which has pivoted research towards improving the efficiency of such models by means of efficient model architectures, dedicated hardware accelerators and model compression. Regarding model compression, a common approach is to remove redundant computations and parameters, referred to as model pruning, and to reduce the numerical precision, referred to as model quantisation. However, most of the current state-of-the-art methods tackle pruning and quantisation separately which might not cover relevant interactions. Moreover, these methods often require additional finetuning of the pruned and quantised model which may not always be necessary. Therefore, a novel method for jointly pruning and quantising a neural network that automatically identifies an optimal policy given the available resources is proposed. Different from the current state-of-the-art is that the proposed method does not rely on additional finetuning and only needs a limited amount of data. As public datasets containing continuous recordings by a WASN were lacking, a novel dataset was collected using a wireless acoustic sensor network in a home environment. The dataset has been made open-source to the research community to aid further development in this direction.

Publication year:2022

Accessibility:Closed