Online Monte Carlo Planning with QoS Subgoals for Data Caching in ITS MEC Networks
Abstract
Multi-access Edge Computing (MEC) is an imperative for next-generation Machine-Type Communications (MTC) to alleviate the shortcomings of Cloud Computing (CC) infrastructures in terms of service delays and network loads. Intelligent Transportation System (ITS) is an application area of MTC that utilizes edge network nodes and Vehicle-to-Everything (V2X) communication technologies to leverage the benefits of MEC resources such as caching popular content data in nearby edge devices and eliminating redundant data fetching operations. This paper tackles the optimal data caching problem in geographically distributed edge networks and proposes Online Monte CArlo planning based data caching (OMCA) scheme for vehicular environments. Considering multi-dimensional Quality-of-Service (QoS) requirements in the ITS domain, OMCA uses Monte Carlo Tree Search (MCTS) al-gorithm with subgoal based temporal abstractions for au-tomatically discovering and optimizing data caching actions. Employing the BeIntelli smart mobility platform of TU Berlin, we compare the performance of our approach with traditional as well as Reinforcement Learning (RL) based methods in terms of cache hit ratio for varying network sizes and cache capacities. Our results show that OMCA utilizes edge network cache resources more effectively compared to other techniques and outperforms MCTS, Deep Q-Learning (DQL) as well as recency based popular caching policies.