Contactless acoustic based indoor monitoring systems: a study on different acoustic models and learning strategies

The number of physical devices connected to the internet has increased significantly in recent years. This resulted in the concept of 'Internet of Things' (IoT) where everyday devices (e.g. home appliances, vehicles, industrial machinery, etc.) become an entity on the internet with the ability to exchange their sensed observations. The combination of a local computing unit to process the collected information together with the ability to communicate over the network results in a 'smart behaviour' and thereby improving the interaction with the end-user. The use of IoT-enabled devices for indoor monitoring applications has already been widely examined in the literature during the last decade. Especially the use of relatively simple IoT sensors such as those that measure temperature, humidity and passive infrared (PIR) gained a lot of research interest. This doctoral research will focus on the use of acoustics as a sensing modality which until now received less attention for its purpose in indoor monitoring applications. Nowadays, acoustic data can be collected from the environment without the need of expensive and sophisticated hardware by simply using the built-in microphones that are available in different types of devices that we already use in our daily lives (e.g. television, home assistants, speakers, etc.). The use of acoustics for indoor monitoring applications has some interesting advantages compared to other sensing modalities. First of all, acoustic sensors can collect information from the environment at a distance and thus does not require any physical contact with the objects that are being monitored. This implies that a single acoustic sensor can be used to monitor multiple objects at once or even an entire room and thereby reducing the required number of sensors significantly. Secondly, acoustic data is highly informative and contains a lot of information about the environment being monitored. For instance, the spectral part of the data gives insight in the type of sound events that have been occurred, e.g. footsteps, speech, opening/closing doors, etc., while the spatial part can be used to extract information about the location of the sound sources. Thirdly, acoustic monitoring is unaffected by change in illumination and does not require a line-of-sight path to the sound sources being monitored. This implies that acoustic observations systems are typically less sensitive to occlusions (obstruction in the line-of-sight path) compared to video cameras or passive infrared motion detectors. This doctoral research has three main contributions regarding acoustic monitoring of indoor environments. This first contribution is situated in automatically classifying of activities of daily living from the available acoustic information in order to support the elderly health care. More specifically, by automatically compiling a summary report about the performed activities over a longer period of time (e.g. few weeks) allow caregivers to make an objective assessment of the health status of the elderly with minimal effort. For instance, increasing activity durations over time or frequently skipping a meal might indicate an underlying health issue. The main focus during this research situated in the use of existing supervised machine learning algorithms which have been evaluated on noise robustness and computational complexity to assess the edge integration in an IoT module. The second contribution of this doctoral research is situated in modelling and classifying of acoustic events using a 'weakly supervised' and 'multi-label' convolutive learning algorithm. The multi-labelling aspect makes that multiple labels (e.g. footsteps, speech, closing door, etc.) can be assigned to a larger acoustic segment (e.g. 30 seconds of sound data). The weakly supervised part implies that not all occurring events in a given segment must be labelled and that no begin nor end timestamps of the individual events need to be provided. This strategy in labelling the data reduces the workload in labelling the data significantly. The proposed algorithm is evaluated on the dependence of unlabelled events per segment, the influence of overlapping events in the training set and the robustness to background noise. The last contribution of this doctoral research is devoted the development of a 'adaptive' acoustic presence detector. The objective of the classifier is situated in discriminating presence from absence related sound data and not in classifying individual sound events from data labelled by user specific inputs (e.g. turning on the lights, opening door, etc.). Furthermore, the computational complexity of the proposed learning strategy is also taken into account such that the presence detection algorithm can be mapped to an embedded platform. The aforementioned contributions will be investigated in this dissertation on self-collected or publicly available real-life acoustic datasets. All examined machine learning algorithms are implemented in a technical software development environment (MATLAB) and are subsequently validated regarding their intended application. The concluding remarks will finalise this doctoral dissertation by briefly recapitulating the research trajectory and the obtained results together with a short discussion about the future research prospects and how these aspects can be tackled.

Publication year:2019

Accessibility:Open