How will generative AI change system development?

Generative AI trends toward automation of system development

Washizaki: As President of IEEE Computer Society, one of the world’s largest academic societies for computing, I am closely watching the trends in generative AI research. While generative AI is being used in a wide range of areas, I believe that questions such as how generative AI will be used in software systems and information systems to create new value, and how generative AI itself will advance, are hugely important themes that we need to address.

Ogawa: The automation of system development by generative AI is being discussed actively. Tools and services like GitHub Copilot that assist with software development, such as coding in a development environment, are coming into practical use. Chat-type AI like ChatGPT is also being used during software development. I’m hearing that software development companies in Japan are each carrying out their own initiatives in this area. First of all, I would like to ask Dr. Motoyama about the situation regarding generative AI use in Hitachi’s IT business.

Motoyama: Currently, Hitachi like others is going ahead with initiatives toward use of generative AI in a wide variety of fields. In the software development field in particular, we are working on provision and deployment of the “Hitachi GenAI System Development Framework,” in which generative AI will assist with all kinds of system development processes, not only design, implementation, and testing but also steps such as defining requirements and project management. We are continuing to release new support functions in this framework and to expand the scope of its use. We are also working on getting generative AI to assist with program migration and testing when carrying out migration and modernization from legacy systems to new ones.

Ogawa: Prof. Washizaki, from a global perspective, what is your view of the usage status of generative AI or of large language models (LLMs)?

Washizaki: According to a survey of literature published in 2024 on LLM use in software engineering, software development in the narrow sense, such as design, implementation, and testing, accounted for the majority of the studies. Among the rest, software management made up around a fourth and quality assurance a sixth, as the scope of LLM use is seen to be steadily expanding.

Washizaki: According to another paper,(*1) investigating the benefits of LLMs, while these models are considered useful in code generation and software testing, they get low marks when it comes to code defect detection, suggesting that there is plenty of room for improvement and advancement. We have a pretty clear idea right now as to what AI can do and what is still difficult.

(*1) Zheng, Zibin, et al. "Towards an understanding of large language models in software engineering tasks." Empirical Software Engineering 30.2 (2025).

Ogawa: So what generative AI is good at and not so good at currently has come into view; but what it can do is growing day by day, as LLM technology advances with astonishing speed. How much progress do you expect to see three years from now in the use of generative AI for system development processes?

Washizaki: During the next few years, for somewhat routine work tasks, I believe AI will demonstrate performance above that of human beings. Even while reaping the benefits of this progress, it will be necessary to consider the risks of reliance on outside generative AI, such as copyright and security issues. One option will be for individual companies, in addition to LLM, to have their own small language models (SLM) specializing in certain kinds of work tasks.

I personally believe that over the next decade, the application of generative AI to tasks relating to requirement analysis, the initial step in system development, should be expanded. Surveying papers published in the past three years shows that applications of generative AI cover a rather wide range of processes, including identification and elicitation of requirements, analysis, specification, and validation. Examples of use in the management area, however, are still few; and research in the security and safety areas should also become important in the future.

Generative AI will facilitate the passing down of knowledge

Ogawa: From a business standpoint, what kinds of expectations do you have for generative AI besides making operations more efficient?

Motoyama: In addition to system development, I would like to see generative AI become able to pass along knowledge of skill holders in areas like manufacturing and maintenance, and fault recovery. That is, by formalizing knowledge that up to now has been accumulated by people experientially, so that it can be handled by generative AI, my hope is that those newly assigned to a job will be able to perform work as skillfully as veteran workers. Passing along that knowledge from veterans to novices, however, will take a massive effort. It will take time first of all to extract tacit knowledge from people; and even if that tacit knowledge is successfully extracted, you still have to solve issues such as how the tacit knowledge accumulated in one specific field can be applied to other operations.

Ogawa: So not only does making operations more efficient help solve personnel shortages, but the passing along of knowledge by generative AI will also address the decline in human resources with high-level knowhow. I see this as highly significant for society.

Now I would like to introduce the future vision currently under study in the Hitachi Research and Development Group.

Many different people are involved in system development, from users to developers and project managers. These various players carry on with the development while exchanging knowledge with each other, and such human-to-human interchanges are a major contributor to cost. What we are now studying is a multi-agent decision-making support capability, whereby multiple AI agents possessing expertise corresponding to each human role make decisions while carrying on dialog, with human beings also taking part in that dialog. We believe that through fast and multifaceted decision-making, development speed and quality will be improved, and knowledge can be passed on to subsequent generations.

Two approaches to the use of AI agents

Washizaki: There are two ways in which AI agents, devising their own procedures and applying the PDCA cycle, can be used. One is having them reliably perform routine tasks; the other is incorporating at low cost the opinions of each agent from their different perspectives. Having AI agents with different characters carry on discussions while repeating the PDCA cycle should be useful toward obtaining higher-quality results.

I am currently engaged in research on automatically generating measures for mitigating threats to AI systems; and here, too, knowledge about security is being expanded through discussion and inferencing among LLMs in different roles.

Ogawa: How exactly do you go about assigning roles to each LLM?

Washizaki: At the present stage, we are simply setting different personas for each agent or changing how they incorporate outside knowledge.

The issue for AI agents comes down to how to equip them with knowledge. Right now we have two approaches, one in which the LLM incorporates a knowledge graph(*2) and one going in the other direction, using the LLM to further enhance a knowledge graph. While an LLM contains general knowledge, it is also a black box, making it difficult to validate the knowledge. The knowledge in a knowledge graph, on the other hand, is limited, but is explicit and easy to validate. It’s important to merge both of these.

We are also conducting research on LLM-knowledge graph merging. For example, one method is first to have the LLM perform inferencing, then incorporate in the knowledge graph the LLM’s method of expression and interpretation, merging them in this fashion. An alternative approach is, while the LLM is performing inferencing, to have an AI agent explicitly search for an outside knowledge graph that can be validated. The merging of LLMs and knowledge graphs is likely to advance further.

(*2) Knowledge graph: A knowledge network systematically linking various knowledge and representing it in graph form.

Ogawa: To what extent has this kind of technology been implemented?

Washizaki: Research is well along on methods of incorporating LLM inferencing in a knowledge graph, like having the LLM read past design documents and make them into formal knowledge; and partial implementation has also begun. The approach of making use of outside knowledge graphs when the LLM is inferencing is still at the research stage, however.

Research on application to software engineering is also progressing. In software engineering, the parts other than coding all come down to operations and other business knowledge. The problem is how to go about making use of this kind of knowledge.

Speeding up the development cycle leads to customer satisfaction

Ogawa: As generative AI continues to advance, we see a time when software development will be sped up with the help of AI, and even a world in which the system enhancement cycle can be completed in one day.

Ogawa: The ideal is to become aware of changes in the user or society during system operation, and based on that awareness, to propose system improvements and implement the necessary system revisions by the following day. When instant development becomes possible, it may be time to focus anew on the original purposes of system development, namely, to improve and speed up customer operations and to create new value.
Assuming it were possible to complete the enhancement cycle in a day, how would that impact business?

Motoyama: The software world is already dealing with a serious personnel shortage, so speeding up the system enhancement cycle would increase the amount of development that can be performed, which I feel would be of enormous value.
At the same time, I think there is also a possibility that a considerable portion of the development processes and work performed up to now may become unnecessary. Currently we are trying to replicate in generative AI the software development-related procedures one by one. It just may be, however, that rather than continuing based on the conventional development processes and procedures, a completely new approach may be viable.

Washizaki: Yes, we are now in a transitional period. The mainstream approach today is to make conventional processes more efficient. Thinking further ahead, as an alternative to the conventional sequence of first thinking about what to build and then developing and validating it, there may also be an approach of starting from development and then thinking about the possibilities. Just the other day we were discussing this in the course of working on a project with Hitachi. This would mean changing to a process of first having AI build something and then modifying the result in line with the customer’s needs.

Ogawa: Up to now, you could spend three months on development and the end result still might not align with the customer’s needs. If you could develop something in a day, those three months could be devoted to devoted to 90 enhancement cycles.

Motoyama: There is a lot about software that cannot be understood just from what can be seen with the eyes. Thinking about requirements after first developing something may not be a bad idea.

Washizaki: A generative AI that can give form to concepts in words close to the customer would, I think, have real value not just for the developer but from the customer’s standpoint as well.

Ogawa: One of the attractions is that by adopting an agile development approach and speeding up the sprint cycle, the opportunities for feedback will increase. In this case, however, the need for decisions by humans will also increase exponentially. Human work styles are likely to change as a result.

Motoyama: Ideally, you would want the decision-making itself to be done by generative AI.

Washizaki: That’s true. If, for example, plans for rebuilding a house came out in a hundred different variations, you would have trouble deciding among them. Even if the AI internally came up with a huge number of patterns, you would want it to show you only the best plans, after narrowing them down.
In software development, there are many things the customer wants that are not implemented due to system constraints. What’s important is to ferret out what has not been done and what the user or customer really wants to do, looking for methods that are feasible. By so doing, I believe it will become possible to fulfill the wish of software development and system development, that of truly grasping and realizing the value to customers and their objectives.

Cross-linking between machine learning and system development to realize a true AI system

Motoyama: While up to now the discussion has been about using generative AI to develop IT systems, today the talk among software engineers is about the possibility of making an IT system with AI built into it.

Washizaki: Conventional software systems are configured according to definitive logic and rules, with a development team responsible for the development, and enhancements are carried out in response to feedback from users. What we now need is a different type of system, with machine learning built in, while performing machine learning based on data. I expect when that happens, it will likely become necessary for the people who deal with definitive logic and entire systems to collaborate with the people who handle data-based LLMs, deep learning, and machine learning.
In a project I am now working on with Hitachi, we are proposing a framework depicting three mountains. (See the figure below.)

On the left, first of all, the “Problem analysis and risk analysis” mountain is for thinking about the overall requirements and risks, while the “Solutions and design” mountain in the middle is for incorporating these in the system design. Then on the right, the “AI training, assessment, and revision” mountain is for assembling data, enhancing its quality, and training and assessing AI. What’s important here is not to stop with the AI mountain as it is, being sure to return to the world of the system as a whole.

There is a tendency when developing machine learning technology to look only at accuracy; but the original objectives need to be dealt with, such as the kind of value it has for the customer and whether it meets the requirements. This is why it is necessary to link between machine learning and the system, based on value.
The situation today is that these mountains are mixed together indiscriminately. I believe interactions across these domains will become increasingly important.

Ogawa: Do you think it is possible for this coming and going between these mountains to take place not just between human beings but automatically?

Washizaki: Yes, exactly. The next step will be to have AI agents that assist with each mountain go back and forth between mountains. This will make it possible to track projects from the standpoints of objectives and value, and also from the standpoint of accuracy. At the same time, I feel it will increase the need for maintaining consistency. The same applies to company organizations. I believe that the IT team, system team, and AI team, which have been mostly discrete entities up to now, will increasingly collaborate with each other.

Ogawa: In Hitachi’s Digital Systems & Services sector, data analysts, business divisions and the like are working on projects as a unified undertaking. Perhaps we could say they are carrying out this kind of collaboration.

Washizaki: That’s a highly interesting approach.

Developing human resources able to validate the results produced by generative AI is essential

Ogawa: In fact, prior to our discussion here, I had ChatGPT write a tentative scenario [laughs]. What it came up with, though, was not at all interesting and I ended up having to do a major revision. Even so, rather than thinking up something from zero, I felt it was faster using generative AI to prepare a rough draft as a starting point.

Washizaki: Yes, that’s the right way to use it, for sure. An LLM being a general-purpose tool means that it suits most people, so if used as is, it will not be interesting. The important thing here is how to deal with this reality.
By using generative AI, even staff with a shallow understanding of operations will become able to generate output as an accumulation of past knowledge. When it comes, however, to intervening when that output alone is not sufficient, and adding new perspectives, the human role will be vital.

Ogawa: In such an era, it will be all the more important not only to increase human resources able to use generative AI, but to develop human resources with essential knowledge about computing and AI, or about the application fields.

Washizaki: What it comes down to is, so long as the problem of generative AI hallucination exists, solid verification will be needed. So again, there will continue to be a need for education and training on the human side. That’s because it would be a real problem if people came to meetings armed only with the raw output generated by an LLM.

Ogawa: Thank you very much, both of you, for taking part in today’s discussion. We’ve covered a variety of topics today, from the relationship between generative AI and software engineering, a vision of the future when generative AI becomes mainstream, and other technology themes, to matters such as organization and human resource development. There are still many other issues needing to be studied on the way to realizing the kind of future discussed today. While solving these issues one by one, I hope we will be able to open up a new field, that of fusion between AI and software engineering, through industry-academic collaboration.

Profiles

WASHIZAKI Hironori

Professor, Waseda University; Visiting Professor at the National Institute of Informatics
PhD (Information and Computer Science)

Other areas of activity include: 2025 President of the IEEE Computer Society, ISO/IEC/JTC1 SC7/WG20 Convenor, Former Secretary of the Special Interest Group on Software Engineering of the Information Processing Society of Japan, Chair of the Software Quality Profession study group committee of the Union of Japanese Scientists and Engineers, and Director of Smart SE: Recurrent Education Program of IoT, AI, and DX.

MOTOYAMA Atsushi

Digital Systems & Services, Hitachi, Ltd.
Head of AI CoE
PhD (Systems Management)

After joining Hitachi, he worked on application development in the industrial and logistics fields. From 2023 he led a project charged with building a platform for generative AI use in the Hitachi Group as a whole. Based on the generative AI use cases and application technologies from that project, he launched a generative AI business and has been providing services across industries.

OGAWA Hideto

Systems Innovation Center, Research & Development Group, Hitachi, Ltd.
Distinguished Researcher
PhD (Information Science)

He is engaged in software engineering research and practical applications. He also works as Industry-academia-government collaboration Visiting Professor at the Japan Advanced Institute of Science and Technology, and as Visiting Lecturer at the University of Tokyo. Among his many activities relating to AI quality, he is vice-chair of the steering committee of the Consortium of Quality Assurance for AI-based Products and Services, and a member of the Committee for machine learning quality management in the National Institute of Advanced Industrial Science and Technology (AIST). He has co-authored numerous books including AI sofutouea no tesuto (Testing of AI software), Sofutouea tesuto no seori (Software testing theory), and Jissen seisei AI no kyokasho (Practical generative AI).