Bayesian optimization algorithm with agent-based supply chain simulator for multi-echelon inventory management

Haiyan Wang

R&D Division, Hitachi America Ltd.

Introduction

Having an efficient and resilient supply chain (SC) is critical for companies to stay competitive, and the disruptions caused by COVID-19 in the global supply chains have re-emphasized the importance of effective supply chain management (SCM). Among various SCM problems, inventory optimization that determines a replenishment policy (i.e., when and how much to order) with balanced inventory costs and service levels, is essential to ensure SC efficiency and increase customer satisfaction.

In practice, however, it is challenging to develop an optimized SC inventory policy. The reasons being, firstly, the SC structures are often complex since manufacturers with global businesses commonly use multiple tiers, or echelons, of stocking locations to minimize logistics costs, inventory costs, and sales opportunity loss. Next, apart from the complex multi-tier SC structure, other factors such as, the highly uncertain and rapidly changing market demand, the uncertain lead time between tiers, the tremendous variety of product types, and much-shortened product lifecycles, further complicate inventory optimization. Such practical factors make a conceptually simple inventory optimization problem, analytically intractable.

To analytically derive managerial insights, researchers have developed classical inventory management theories based on simplified assumptions such as static SC structure and stationary demand. These insights have been used in practice as the basis for a wide variety of heuristics but often result in sub-optimal SC performance because of violations of assumptions. To improve SC performance and solve analytically intractable real-world SC inventory optimization problems, researchers and practitioners have turned to simulation-based optimization approaches. Simulators, however, have traditionally been developed to solve a specific problem under consideration and are therefore not flexible or comprehensive enough to accommodate new dynamics such as SC structure changes mandated by the highly dynamic global market and increasing competition. Another challenge that hinders the practical usage of simulation-based optimization is that they are usually computationally expensive. This is because the deployed meta-heuristic optimization procedures such as the generic algorithm (GA) require many parameter evaluations when searching for the optimal SC parameters.

To enhance the applicability of the simulation-based optimization method in solving real-world SC inventory optimization problems, we developed Bayesian optimization approaches along with a flexible and comprehensive agent-based SC simulator. Numerical experiments have shown that our methods are much more efficient than popularly used GA.

Hitachi technology for supply chain inventory optimization

Our technology consists of two main components: an agent-based simulator that can simulate complex dynamics and characteristics in a real-world SC and an efficient Bayesian optimization framework, as depicted in Figure 1. Before we describe the main feature of each component, let us look at one example mathematical formulation of the SC inventory optimization problem.

• Problem formulation
In practice, various inventory policies are used, for instance, periodic order-up-to policy, forecast-based days-of-supply, etc. For a given inventory policy, we need to optimize the corresponding parameters to minimize the total expected systemwide inventory cost while maintaining a minimum demand-fulfill rate. For example, we need to optimize the order-up-to quantity level in an order-up-to policy, the number of look-ahead days in a days-of-supply policy. That is, the inventory optimization problem can be formulated as the following constrained optimization problem:

• Agent-based SC simulator:
The agent-based SC simulator developed by our team *1 can be rapidly customized to build a SC digital twin for any real-world SC. Moreover, it has the flexibility to model all commonly used inventory policies and to accommodate situations when the SC structure changes, when the SC managers change the inventory policy structure, when the market-demand pattern changes, etc.

The SC digital twin that can accurately replicate the physical SC process and activities is used to estimate the relationship between SC performance measures (i.e., the expected inventory cost and the expected demand fulfill rates in the above formulation) and the inventory policy parameters. To account for the inherent uncertainties such as demand and transportation time between tiers, we also developed a scenario generator outside the SC simulator. This modularized design dramatically improves the flexibility and generalization of our simulator. The scenario generator can learn from historical data and then generates different scenarios that serve as input to the simulator to help accurately estimate the SC performance metrics to support optimal and robust inventory optimization.

• Bayesian optimization for multi-echelon inventory optimization:
We propose to use Bayesian optimization (BO), a powerful global optimization framework for optimizing expensive and black-box objective functions. Compared with other general optimization frameworks, BO often significantly reduces the number of function evaluations. The fundamental idea of BO is to approximate the objective function by a surrogate model that can not only accurately approximate the objective function but also can be relatively cheaply evaluated at a sample parameter setting.

Note that we need to solve a constrained optimization problem as illustrated in the problem formulation. We experimented two different procedures to tackle the constraint challenge: one is a penalty-based method (PBO), and the other is the constrained Bayesian optimization (CBO) approach *2. CBO outperforms PBO because of its ability to accurately approximate both the objective function and the constraint function, as illustrated in Figure 2. Please refer to our paper *3 for more details.

We conducted an extensive set of experiments to benchmark the model performance against the state-of-the-arts. The optimal inventory cost and the computational time for different methods are given in Figure 3. The proposed approach achieved the best performance in terms of optimality and computational efficiency. Remarkably when the number of demand scenarios N=50, CBO converged to the optimal solution twenty-two times faster than GA. More results can be found in our paper *3 and our extended journal paper *4.

Conclusions

We developed a new high-performing Bayesian optimization-based approach along with an agent-based supply chain simulator to solve the challenging constrained multi-echelon inventory optimization problems. The key features are:

For more details, we encourage you to read our paper, Kiuchi et al. *3.