News release overview
Hitachi and the University of Tokyo develop dynamic pruning technology that speeds up Big Data retrieval time by up to 135 times
Supporting traceability in the manufacturing industry and data utilization in the medical care and finance fields will help solve issues confronting society
Figure 1. Typical application of the developed technology (traceability query example: exhaustive production inspection)
Tokyo, June 19, 2025 Hitachi, Ltd. (TSE:6501, "Hitachi"), and National University Corporation, The University of Tokyo (“the University of Tokyo”), toward speeding up Big Data analysis, have developed dynamic pruning*1 technology that significantly improves the retrieval time of data having complex interconnections (hereinafter, “graph-structured data”*2). The conventional method of performing data analysis in a database, known as recursive query processing,*3 involves successively traversing graph-structured data. Since this approach necessitates repeatedly reading data not relevant to the query, the retrieval speed is slowed down. The newly developed technology, based on information obtained through recursive query processing, accurately identifies in real-time the scope of the next data to be read and skips the reading of unnecessary data, significantly improving retrieval speed. In the application of verification testing to product shipment decisions in the manufacturing industry, data retrieval speed was confirmed to be up to 135 times faster*4 than with the conventional methodology. This speed gain is expected to contribute toward improving the quality of traceability*5 by enabling faster analysis of graph-structured data, such as in tracing processes or parts from product design to manufacturing, distribution, and maintenance (Figure 1).
Going forward, Hitachi and the University of Tokyo will aim to apply this technology beyond the manufacturing industry, such as in predicting disease risk by analyzing medical care patterns in the social insurance sector, or in detecting unauthorized access in the financial sector, thereby promoting technical innovation to solve societal problems.
The results of this development project are to be presented in part at the 2025 ACM SIGMOD/PODS International Conference on Management of Data, an international database conference, being held from June 22 to 27, 2025, in Berlin, Germany.*6
*1 Pruning: A method of speeding up queries by skipping data unnecessary to the query when executing a query.
*2 Graph-structured data: A structure using nodes and edges to represent data. Nodes represent the objects of data, while edges represent the relationships or connections between nodes.
*3 Recursive query processing: In a database or program, a method of solving a problem by dividing it into small parts and successively solving each part. This method is used to trace graph-structured data, particularly for obtaining data with parent-child relationships or a hierarchical structure.
*4 Compared to Hitachi’s existing technology.
*5 Traceability: The state of having recorded the manufacturer, supplier, vendor, and other information about a product at each process from procurement of raw materials and parts to production, distribution, sales, and maintenance, and enabling this history to be traced.
*6 Norifumi Nishikawa, Akira Shimizu, Akira Ito, Shinji Fujiwara, Yuto Hayamizu, Masaru Kitsuregawa, and Kazuo Goda. 2025. Dynamic Pruning for Recursive Joins. In Companion of the 2025 International Conference on Management of Data (SIGMOD-Companion ’25), June 22–27, 2025, Berlin, Germany. ACM, 15 pages.
