Predictive maintenance in industry to increase availability and energy efficiency using multimodal analysis models and agent-guided mobile sensor technology (WISES)

Project description

The innovation of the research project is the real-time detection of anomalies in machines and systems using mobile sensor technology and associated multimodal analysis models, which continuously record and evaluate the signatures collected. AGVs will move through the production halls and record sensor data in order to detect anomalies (e.g. failures, energy losses, leaks). The project is developing novel algorithms that calculate trajectories for the AGVs and recording vectors for directional sensors (e.g. directional microphone, infrared temperature sensor). The collected multidimensional data streams include time series with spatial offset of different sensors. These include, for example, heat emissions, machine load, power consumption and air flows. Analysis models not only classify system behavior and detect anomalies, but also provide feedback for planning trajectories and recording vectors to maximize classification quality. Another significant innovative approach is the development of a comprehensive speech-based recommendation system that can support predictive maintenance using historical data and the results of analytical models, while keeping the human in the loop. Context-based recommendation is a self-updating technology that uses state-of-the-art language models (e.g. Llama 3) that understands both text and voice data and can be communicated remotely by maintenance experts via the cloud. One specific example is the detection of compressed air leaks in printing plants. Detecting such leaks leads to massive energy savings and prevents subsequent damage and downtime in the print shop. This represents a significant innovation compared to the state of the art, which predominantly relies on stationary sensors or human maintenance technicians. Stationary sensor technology has significant disadvantages and limitations in terms of scalability, flexibility, adaptability, cost efficiency and multifunctionality. Human maintenance technicians are not only cost-intensive, but also lead to high time requirements and limitations in terms of the types of anomalies that can be detected. In addition, production facilities such as printing plants are not permanently suitable for maintenance technicians due to the environmental conditions (e.g. noise levels). WISES is developing a mobile, AI-based sensor system for detecting anomalies such as leaks in industrial plants. Autonomous sensor robots (AGVs) record heat, noise and vibration data and evaluate it in real time. It increases the availability of machines and helps to save energy.

Focal points of the project

• Development of an agent-guided, mobile sensor system (AGV) for the multimodal recording and analysis of industrial conditions.
• Real-time detection of anomalies through combined evaluation of heat, sound, vibration and image data.
• Use of Large Language Models (LLMs) to derive context-sensitive recommendations for action for maintenance technicians.
• Establishment of a feedback loop between sensor technology, analysis and recommended actions for the continuous improvement of maintenance.