As a student, gaining the perspectives of user interface and system design
Morimoto: I am a product of a system engineering department in a faculty of engineering, a department where students tend to know a lot about computers. In my senior year, I learned from a faculty member who came from outside the university about the user interface field, a field for designing the place where human and machines intersect; and since that time I have maintained my interest in this field. When I joined Hitachi after graduation, my research centered on natural language processing and Japanese search technology in the information science field, that is, algorithm research. I became aware that even more important than outstanding search algorithms is thinking about what human beings are looking for. So I shifted the focus of my studies to the upstream, or the overall system, such as how to design the screens and how the functions are used in actual work processes. With natural language processing technology as the base, I continued to research its use in call centers, internal company systems and other business systems. Writing a number of academic papers, I was able to earn my doctoral degree (while at Hitachi).
Ikegaya: While in junior high school, I came up with my own geometric proof problems and solved them myself. I was fascinated with the way figures could be expressed by mathematical equations. Later I started thinking about what kind of future career could result from somehow combining mathematics and art. I had a relative who was an industrial designer, and my art teacher in high school was a former industrial designer. So when I was a high school student, I decided I too would become an industrial designer in the future. At the time, product design was the mainstream, and I intended to pursue that field as well. In university, however, my professor told me that the importance of product design was diminishing, and that the kind of design in demand was in systems and services. This was something of a shock to me, but at the same time it definitely sounded interesting; so when I heard that Hitachi was conducting research on the design of social systems, I followed my interest and joined the company. Although my first year was spent designing mobile phones, which was very much product design, during my second and third years I gradually became involved in system and service design. From around 2010, while engaging in service and system design, as well as being involved in the Catch Kizashi (future signs)* project and “customer co-creation,” I was able to take part in many Hitachi fields, such as energy and mobility, while creating graphic recording demos and so on.
*Examples of Catch Kizashi (future signs);
Mediating between design and the environment, on to exhibiting at COP26
Morimoto: As my own research was gradually gaining an outlook on the upstream and systems as a whole, in 2015 the Research and Development Group was reorganized, resulting in my working on the same designer team with Ikegaya-san. That came as quite a shock. I never imagined I would have the experience of sitting next to someone, in the same room, who didn’t speak the same language. (laughs)
What I became involved in thereafter was a value simulator (Cyber-Proof of Concept; “Cyber-PoC”;). While researchers are keen to make customers aware of how wonderful their technology is, customers tend to be mainly interested in the extent to which the technology will serve their needs. We therefore hit on the idea of using information technology to visualize for customers the kind of value the technology can provide them. We created a framework in which researchers developing the algorithms for simulating value worked together with the designers responsible for devising the user interface, that is, the method of showing the results so that stakeholders can understand them.
Ikegaya: After working with Morimoto-san, I was stationed in the UK, where my awareness of the environment greatly changed. I got the impression that they were four or five years ahead of Japan, and residents had a high awareness and sense of crisis. In fact, one of the reasons I joined Hitachi was for the “social contribution,” and I myself wanted to become involved in environmental activities; so I was lucky in being assigned to a team working on the environment upon my return to Japan in 2019. Hitachi, meanwhile, became a principal partner of the 26th Conference of the Parties to the United Nations Framework Convention on Climate Change (COP26), held in Glasgow in November 2021. While this was in the midst of the COVID-19 pandemic, I was one of five people from Hitachi in Japan able to participate in COP26.
Morimoto: In 2021, the two of us happened once again to work alongside each other. His mission was to raise the presence of Hitachi in the environmental field, whereas I was studying the spread of value provision through Cyber-PoC.
Ikegaya: Wanting to exhibit at COP26 making use of Hitachi’s latest technology, I noticed Morimoto-san’s Cyber-PoC. It was also right at that time that I myself was engaged in the preparation of BEYOND SMART LIFE, a work summarizing the accomplishments of the Hitachi Kyoto University Laboratory; and that is where I learned about research on the use of AI for policy recommendation. This “policy recommendation AI” research was about using AI to simulate the various possibilities that might arise, say, thirty years from now, and the many future scenarios for arriving there as well as the scenario critical (branch) points. I thought it would be interesting to link this to Cyber-PoC. This was the memorable first step of the decarbonization scenario simulator, right?
Morimoto: That’s right. When I heard about this idea from Ikegaya-san, I thought it was quite exciting. The Cyber-PoC mission, aimed at business operators, is to visualize value a few years in the future, whereas the environment is a theme with the year 2050 in its sights. What this means is that the need will come to think about how to express value to people in the future, and how to convey scenarios to people today. Thinking that it might be possible to raise the Cyber-PoC technology to a whole new stage, I was sure I wanted to work together on this.
The decarbonization scenario simulator as a tool enabling anyone to be a data scientist
Ikegaya: The technology on which the decarbonization scenario simulator is based is the policy recommendation AI developed by the Hitachi Kyoto University Laboratory (https://www.hitachihyoron.com/jp/archive/2010s/2019/03/05c04/index.html). One result emerging from this research was a causal model for around 150 social factors, such as the declining birth rate and environmental degradation. Thereafter, by means of AI-based simulation, it became possible to predict approximately 20,000 future scenarios over a 35-year span from 2018 to 2052. In this case study, these were classified into 23 representative scenario groups, indicating future possibilities. The simulator further analyzes when and how scenario groups branch from each other and arrive at their outcome, thereby assisting with policy recommendation.
Starting in 2017, many initiatives were carried out in relation to the policy recommendation AI. One example is a joint research project with the Fukui Shimbun, in which Fukui Prefecture residents were asked for their views on what leads to happiness; and workshops were held in which the policy recommendation AI was used to create recommendations leading to happiness in Fukui. I myself was not involved in this, but I thought it was an interesting way of using this AI. At the same time, given the way discussions were carried out by going through steps with AI support, I felt it had potential as a consensus-building tool, and that it could be used as is in decarbonization initiatives.
Morimoto: While the simulation itself is handled by AI, I projected that the behavior and awareness of residents could be changed by using the results to get them actually thinking about these matters. Going beyond something like Cyber-PoC, aimed at corporations, which provides value to sponsors who pay for the service, I felt it should become a tool for engaging the public in dealing with issues of the environment.
Ikegaya: One issue, though, was that the simulation results indicated by the policy recommendation AI were shown as rows of numbers, which effectively meant these had to be interpreted by the AI researchers and data scientists. When it came to conducting simulations of decarbonization scenarios and exhibiting at COP26, I saw the need to develop a user interface that would express these figures graphically so they could be understood by the general public. After consulting with Morimoto-san, I embarked on development of a web application that could display the results graphically and that anyone could use.
Morimoto: When the simulations are run, it becomes possible to analyze branch points (critical points) such as, “this is the only future outcome if we proceed on the present course,” or “we cannot arrive at this scenario without making this kind of decision.” I carried on many discussions with designers and other experts about expressing these graphically so they could be readily understood. In the simulations, 23 scenario groups are generated, but the scenario branch points are complex and hard for non-specialists to grasp. With the decarbonization scenario simulator, we therefore decided to consolidate into around six to eight large scenario groups with 2050 as their arrival point, and to color-code them.
Ikegaya: Black-and-white charts had been used up to that time, so just color-coding the major groups greatly enhanced the visibility.
Morimoto: Very much so. Tracing back each of the colors from 2050 to the present time, at a certain point the colors branch. For example, you can see the critical point where the division into red and green scenario groups takes place. To make things even easier to understand, we created screens for confirming the scenario branches in detail. We put pie charts at the branch points, and color-coded the charts to make intuitively clear the likelihood of getting to the scenarios of each color beyond the branch point. In case it was too late by the time the branch point was passed, we made possible a do-over, performing further simulation from that point so as to approach the target scenario.
Ikegaya: We need to have the general public talk about their future and do data science. To enable this, I had an interface made that would be as easy to understand as possible. I came up with the basic idea of the user interface, but the actual detailed screen designs were developed by the designers on Morimoto-san’s team, so that the final interface is a fruit of collaborative creation. The reason for making a web application was that I wanted the decarbonization scenario simulator to be usable by anyone in the future even without intervention by Hitachi. Knowing what the previous interface was like, I’m impressed by how far along we have come (laughs), but there are still many features that have yet to be implemented. We plan to continue enhancing the functions.
Preparation of a decarbonization roadmap in two Hokkaido municipalities
Ikegaya: Testing of the web application developed for the decarbonization scenario simulator was conducted in collaboration with two municipalities in Hokkaido to verify its value. The verification took place in the cities of Obihiro and Ishikari. Workshops were held, with the local people playing central roles, on building models for the AI simulation, interpreting the scenarios for 2050 sought by AI, and considering possible roadmaps.
Morimoto: What’s crucial here is for the local people themselves to create the models for analysis by AI. Having the people who know their issues create a model is the best way to ensure the resulting scenarios are matched to reality. While decarbonization is the aim, the circumstances in each municipality, such as whether the population is growing or declining, the industrial structures, and fiscal resources, are different; so it goes without saying that issues vary greatly depending on regions or municipalities.
Ikegaya: Generally, the bigger the scale of a municipality, the more difficult it becomes to build a consensus. In a small municipality, the mayor and small assembly can come to a conclusion in a short time; but with a bigger population, the stakeholders increase proportionally. I would guess that the effectiveness of consensus-building using the decarbonization scenario simulator is best seen in a city of from 100,000 to 500,000 people. The workshop participants included not only those directly involved in the environment and decarbonization, such as in the environmental department of their municipality, but people from other departments as well. We also invited participation by citizens' groups and NPOs. In Obihiro, students from the local University of Agriculture and Veterinary Medicine took part, as we succeeded in having issues reflected from the viewpoint of residents.
Morimoto: The workshops were held four times in each city. The first session was used to identify local issues and indicators, and study causal models. In the second session, the causal model was further refined, and based on the result, the decarbonization scenario simulator AI analyzed scenarios. At the third workshop session, while examining the scenarios derived by AI, each scenario was assigned a name based on an interpretation of their characteristics. Then in the fourth session, the scenarios to be aimed for were selected, and while examining the indicators to be adopted at each critical point for realizing a scenario, these were reflected in a roadmap.
Ikegaya: We talk about “aiming” for decarbonization, but there are many directions toward that end. For communities to continue existing, their populations must grow; but if the new members all get around with cars running on gasoline, their carbon emissions will increase. So can they just use EVs (electric vehicles) instead? That would mean higher electricity bills, and generating that electricity may result in higher carbon emissions as well. The relevance of around 300 indicators was discussed in the workshop and they were systematically arranged. The AI presented scenarios right for that community, and then on this basis the scenarios were compared and an appraisal was made as to which should be selected.
Morimoto: In this verification trial of the decarbonization scenario simulator, a wide range of local issues were raised, not limited to certain departments, and we were able to have discussions on indicators besides those relating to the environment. I believe that’s all the more reason the participants can treat the ultimately derived scenarios and policy measures as very much their own matters. In the end, it’s important to have the people affected make their own choices. Rather than forcing the AI results on them, it is vital that they be asked which to choose and feel right about their choice.
Ikegaya: Each of the scenarios brings them closer to decarbonization, but it’s up to the municipality to decide whether to give priority to economic factors or to the social environment. I see it as giving people the occasion for thinking on their own about the vision for the future they prefer.
Morimoto: Developing the web application, we were able to show the simulation results in quite easily understandable form. Even then, with around 300 indicators shown, it is still rather difficult to confirm the scenarios from now to the future and their critical points. Given the limits to what can be done with the web application alone, we have been studying combining this with something like PowerPoint or an online whiteboard. There were also many times when we carried out studies while physically affixing sticky notes on a whiteboard. We would like to make further refinements to the user interface of the decarbonization scenario simulator, in the hopes of helping to solve problems facing society.
Yukiko MORIMOTO
Awakening to the user interface that mediates between people and machines
The first book I read after joining Hitachi, Designing the User Interface by Ben Shneiderman et al., is connected to my present work. My interest in the field of user interfaces, which mediate between people and machines, and which I became aware of as a student, is what prompted me to join Hitachi. When operating an air conditioner, for example, and it’s currently 25°C but you want it to be a bit cooler, if you set it to 24°C or 23°C, your objective will be achieved. But when people feel hot, they may instinctively lower the temperature setting all the way to 18°C. This kind of difference exists between people and machines, and in fact, the user interface fills in this gap. This is the way of thinking I learned from reading the book. After joining Hitachi, my research started out in natural language processing; but its base includes user interface studies, which I feel turned out to connect to my present work.
Kazuhiro IKEGAYA
Coming into contact with the collective intelligence of the Hitachi Kyoto University Laboratory to find breakthrough solutions to society’s problems
My involvement in preparing the book BEYOND SMART LIFE (August 2020, Nikkei Business Publications), which was edited by the Hitachi Kyoto University Laboratory, ended up having a major influence on me personally. Besides having professors at Kyoto University write essays for it and review the contents, I contributed to it as well. As a book for finding breakthrough solutions to the problems faced by society on the way to 2050, I feel it is a major source of collective intelligence, and urge everyone to read it. Another work I would like to recommend is Never Out of Season by Rob Dunn (March 2017, Little, Brown and Company; Japanese translation by Hiroshi Takahashi for Seidosha). The author points out that human beings depend on 12 main food groups for our existence, and there is a risk that some of these will disappear due to environmental influences. It is a work sure to spark interest in what we put into our mouths.