Leveraging reinforcement learning for addressing the three-dimensional bin packing problem, applied to a mixed-palletizing use case

Flanders Make

Flanders Make is the strategic research centre for the manufacturing industry. Our mission is to strengthen the long-term international competitiveness of the Flemish manufacturing industry. That’s why we work together with SMEs and large companies on pre-competitive, industry-driven technological research, resulting in concrete product and production innovation in the vehicle industry, the manufacturing industry, and production environments.

Problem Context

The mixed palletizing use case states the problem of optimally stacking assets of various dimensions and weights on top of a pallet. Optimality here is mainly considered regarding geometric stability and volumetric stacking density [1], i.e. stacking items on the smallest number of pallets while establishing a firm construction and retaining the global center of mass centered. In a mathematical context, this problem categorizes within the three-dimensional bin packing optimization problems (“3DBPP”). In literature, the introduced problem statement is often referred to as the Distributor’s Pallet Loading Problem (“DPLP”) [2], referring to use cases within the application area of distribution and logistics where the physical parameters of processed items are not necessarily known in advance, yet these assets need to be stacked on pallets as efficiently as possible in terms of occupied volume for transportation purposes [3]. By the variability of items, the DPLP distinguishes itself from the Manufacturer’s Pallet Loading Problem (“MPLP”), where all items to be stacked on a pallet have the same dimensions and weight, and often intuitive configuration wizards are offered by industrial integrators and robot manufacturers [4]. However, the MPLP is only a particular case of the DPLP, and the latter generalizes and inherently solves the former. Therefore, the addressing of the DPLP provides market potential, not only in logistics and distribution, but also in manufacturing by providing a generalized solution method and eliminating the need for configuration that is still an essential non-automated step to day.

Many researches address the solution of the 3DBPP: either utilizing exact methods [5] [6], heuristic algorithms [7] [8] or reinforcement learning [9] [10]. It is the latter method – reinforcement learning (RL) – that is the highlighted method in the scope of this Master’s thesis proposal. Furthermore, we target to consider an online version of the stacking subject, meaning that the knowledge of future assets to be considered is rather limited relative to the number of boxes stacked on a pallet, and optimal stacking locations need to be rationalized in an online fashion. The literature oriented part of the Master’s thesis targets to identify appropriate methods to address the online 3DBPP in a DPLP context, and to elaborate on the results that were obtained in relevant research around similar topics.

Goal of the thesis

In recent research activities at Flanders Make, a Heuristics based algorithm was developed addressing the problem of interest. Around the algorithm itself, a palletizing framework was developed, serving as an experimental environment to benchmark the performance of different algorithms, and anticipating industrial adoption. The practical part of the Master’s thesis targets to develop an implementation building on top of the existing implementation efforts by creating an innovative RL-based component as part of the palletizing framework. The simulation environment set up at Flanders Make provides a platform to develop, test and benchmark the results provided by the RL-based solver for the stacking problem. Furthermore, a physical scale model setup was built at Flanders Make where an industrial robot arm palletizes boxes that are supplied on a conveyor belt in order to practically validate and showcase the results provided by the algorithm implementations. Both the simulation and the physical setup will be available for validation of the results developed in the scope of the Master’s thesis, and will be supported by Flanders Make researchers.

Profile

• Bachelor degree in mechanical, control, automation, software, AI or mechatronic engineering;
• Has experience with or a strong interest in AI and Machine Learning;
• Basic knowledge of robot kinematics;
• Python is the preferred software language, but the basics can be learned;
• Passionate by research and new technologies with focus on applications for machines or mechatronic systems of the companies;
• Result oriented, responsible and proactive;
• Eager to learn and a team player.

This assignment can only be executed by a thesis student from a Belgian university.

Offer

• Thesis: This assignment is a topic for a master thesis for a Belgian university;
• Follow up moments take place at the offices of Flanders Make located in Leuven, Belgium.