Leveraging audio and time-series sensor data through natural language to help respond to anomalies and transfer skills
Hitachi has developed physical AI technology for equipment maintenance that enhances the efficiency of diagnostic work, helping reduce the risk of equipment shutdowns caused by overlooked anomalies and supporting the transfer of skills and expertise. At equipment maintenance worksites, workers identify signs of anomalies and analyze their causes based on data such as equipment sounds, temperature, and vibration. However, differences in sounds can be difficult to identify in noisy environments, while finding similar waveforms from vast amounts of time-series sensor data*1 can also be challenging. The newly developed technology uses natural language*2 to support both preliminary diagnostics using audio data and detailed diagnostics using time-series sensor data. Specifically, by combining technology that explains the differences between normal and anomalous sounds using natural language with technology that retrieves waveforms described in natural language from a time-series sensor database, the technology makes it easier for even less-experienced workers to understand the basis for a diagnosis. This facilitates the sharing and utilization of knowledge and experience at worksites, helping establish a foundation for stable operation and quality maintenance at manufacturing sites and across social infrastructure. It will also contribute to the realization of physical AI technology that can be readily applied in equipment diagnostics and support the transfer of skills and expertise.
*1 Time-series sensor data: Equipment data, such as temperature, vibration, pressure, and current, continuously collected over time.
*2 Natural language: Vocabulary and expressions that people use in everyday life. In the context of this technology, it refers to the language used to describe equipment condition or characteristics.
Figure 1: Overview of challenges that equipment maintenance worksites face and the support that the new technology provides
Background and issues
Amid ongoing declines in the working-age population, equipment maintenance worksites are grappling with the challenge of streamlining diagnostic work while maintaining the quality of maintenance with limited personnel. However, identifying signs of equipment anomalies and analyzing the causes behind them requires specialized knowledge and experience in interpreting data such as the sounds, temperatures, and vibrations that equipment generates. Given these conditions, diagnostic work has tended to rely heavily on the knowledge and expertise of experienced workers. For less-experienced workers, the tasks of checking sounds that can be affected by the worksite environment, and searching through vast amounts of sensor data to locate the information required to perform a diagnosis can be a significant burden. Accordingly, there is a need for technology that makes it easier for maintenance workers to understand the rationale for decision-making regardless of their level of experience, enabling more efficient diagnostic work and supporting stable equipment operation.
Features of the technology and solutions developed to solve these issues
To address these issues, Hitachi developed physical AI technology that supports diagnostic work for equipment maintenance. The technology uses AI to analyze information from equipment at worksites, including audio data and time-series sensor data, and supports a broad range of diagnostics—ranging from preliminary to detailed—via natural language processing. The natural-language capabilities also include technology that understands the intent of diverse spoken instructions (Figure 2(1)), which had previously been difficult to achieve. The key features of the technology are as follows.
1. Technology to support preliminary diagnostics using audio data
Hitachi developed anomalous sound detection technology (Figure 2(2)) that has been trained to identify the characteristics of both equipment sounds and background noise, making anomalous sounds easier to detect even when there are differences in the noise environments and equipment between worksites. It also developed audio difference captioning technology (Figure 2(3)) that explains the subtle differences between normal and anomalous sounds in easy-to-understand natural language. In addition to easing the challenges of checking for anomalies at worksites, the technology enables workers to understand the differences that form the basis for the diagnosis. By also making it readily possible to verbalize and share the points that experienced workers focus on during diagnosis, the technology supports the transfer of skills and expertise.
2. Technology to support detailed diagnostics via time-series sensor data
Hitachi also developed technology that retrieves locally similar waveforms from large historical databases of time-series sensor data based on descriptions in natural language (Figure 2(4)) and technology that generates training data for AI models based on natural-language instructions (Figure 2(5)). This makes it easier for workers to identify and refer to relevant past cases, even when they have not specified in advance the shape of the waveform or the conditions under which it changes. In addition, workers can easily supplement data for training diagnostic models—even for equipment that has only a small number of anomalous data samples available—which helps facilitate cause analysis.
Figure 2: Example of how spoken instructions can be used to perform both preliminary and detailed diagnostics
Looking ahead
Going forward, Hitachi will conduct PoC trials at equipment maintenance worksites to verify the effectiveness of this technology in performing both preliminary diagnostics using audio data and detailed diagnostics using time-series sensor data. Through the development of these technologies, Hitachi aims to put into practical use equipment diagnostic services that support more efficient diagnostic work and the transfer of skills and expertise, help reduce the risk of equipment shutdowns caused by overlooked anomalies, and contribute to stable equipment operation. Hitachi will continue to leverage its strengths in IT, OT, and products to support innovation in maintenance operations through research and development and practical application of physical AI technologies for equipment maintenance.
The anomalous sound detection technology, audio difference captioning technology, time-series retrieval technology, and time-series generation technology developed by Hitachi are scheduled to be presented at the European Signal Processing Conference (EUSIPCO) 2026, which begins on August 31, 2026. The spoken language understanding technology, which interprets the intent of a wide range of spoken instructions from workers, is scheduled to be presented at INTERSPEECH 2026, the Annual Conference of the International Speech Communication Association, which begins on September 27, 2026.
For more information, use the inquiry form below to contact the Research & Development Group, Hitachi, Ltd. Please make sure to include the title of the article.
https://www8.hitachi.co.jp/inquiry/hitachi-ltd/hqrd/news/en/form.jsp
