CUMULUS: 3D object recognition, localization and activity monitoring based on point clouds (CUMULUS)

A wide range of 3D sensors are currently commercially available, and their price level has considerably lowered in the past years. Sensors based on principles such as LIDAR, Time-of-Flight or structured light capture their environment as a 3D point cloud. Such a 3D point cloud contains a lot of information about the neighborhood, which can be interpreted for tasks such as object detection and recognition. 

Both 3D sensors to capture point clouds and software tools and algorithms to process such point clouds are available in many flavors, and are used in a wide variety of applications. In this project, we will analyse the available technology and apply it to a set of case studies provided by the participating companies. We will look in particular at object recognition, activity recognition and localisation using 3D point clouds. 

We will study the following questions: 

• Which extra opportunities do point cloud sensors provide, as compared to regular sensors? What is their price level, robustness and maintenance cost? What kind of calibration procedures are required?
• Which software is available to process point clouds captured by 3D sensors into relevant information such as localisation, object recognition or map building? What is the precision and reliability of this information in practical situations such as rain, fog or interference?
• Which hardware do I need?
• The sensors and software tools will be applied in a few case studies in the domain of maritime, logistics, agriculture and manufacturing industries. 

In this project, we made an overview of commercially available 3D point cloud sensors and (open source) software libraries for 3D point cloud processing.

In close alignment with the participating companies, we also performed various case studies showing the potential of point cloud sensors in industrial applications. Some examples of case studies are given below.

• 6D object pose estimation and tracking in a production environment
• 3D person detection for safety monitoring in a production environment
• sorting items on a conveyor belt using an RGB-D line scanner
• localization and mapping on inland waterways using a sensor box integrating lidar, GNSS and IMU sensors
• semantic point cloud segmentation of lidar data
• obstacle detection on the road using solid-state lidar point cloud data
• road marking detection in lidar data on roads and airports

See our project website (https://iiw.kuleuven.be/onderzoek/eavise/cumulus/home) for more information.