Master Thesis Internship - Hybrid Vertiport Arrival Manager: Reinforcement Learning Optimization with Rule-Based Safety Supervision

Job description

Background

Urban Air Mobility (UAM) and large-scale drone operations will require automated traffic management systems capable of safely sequencing large numbers of aircraft at vertiports. Similar to arrival managers (AMAN) used in conventional Air Traffic Management (ATM), such systems must determine conflict-free arrival sequences while respecting operational constraints such as separation minima, route availability, pad capacity, and vehicle energy limitations.

Existing concepts often rely on deterministic rule-based algorithms that guarantee safety and predictability. While these approaches are robust and certifiable, they may become suboptimal in complex traffic situations where many aircraft compete for limited vertiport capacity.

Recent advances in Reinforcement Learning (RL) suggest that learning-based methods may improve traffic flow efficiency by discovering optimized sequencing and control strategies. However, purely learning-based systems raise concerns regarding safety, explainability, and certification. A promising approach is therefore a hybrid architecture in which a centralized reinforcement learning-based optimization layer proposes actions while a deterministic rule-based layer acts as a filter to guarantee safety and constraint compliance.

In this centralized architecture, the RL optimizer aims to improve operational efficiency (e.g., throughput or delay), while the safety supervisor ensures that operational constraints such as separation minima and route feasibility are never violated.

This thesis will investigate such a centralized hybrid arrival management concept for drone/UAM operations at vertiports using the BlueSky open-source air traffic simulation environment.

Proposed Research Questions

The thesis will explore several research questions related to hybrid arrival management architectures:

• How can a centralized hybrid architecture combining reinforcement learning optimization and deterministic safety logic be designed for vertiport arrival management?
• What state representations best capture the relevant traffic information (e.g., aircraft positions, ETAs, battery state, pad occupancy) for learning-based optimization?
• What action space (e.g., speed adjustments, sequencing changes, route selection, pad assignment) is most effective for the RL component?
• What type of reinforcement learning algorithm is most suitable for this problem (e.g., Proximal Policy Optimization (PPO), Soft Actor-Critic (SAC), or other approaches)?
• How should the rule-based safety supervisor be structured to guarantee separation and operational constraints while allowing optimization?
• Does the hybrid approach improve vertiport throughput, delay, or energy efficiency compared to purely rule-based sequencing?
• How does the approach scale with increasing traffic density and number of arriving aircraft?
• What are the implications of learning-based decision-making for safety, interpretability, and potential certification?

Tasks

The student will design and evaluate the hybrid arrival management concept through simulation and analysis.

1. Literature Review
   Review existing research on UAM traffic management, vertiport arrival sequencing, and reinforcement learning applications in transportation or air traffic control.
2. Simulation Environment Development
   Implement a vertiport arrival scenario in the BlueSky simulation environment, including predefined routes, arrival traffic flows, vertiport pads, vehicle energy models, and wind effects.
3. Baseline Arrival Management Logic
   Implement a centralized rule-based arrival manager capable of sequencing aircraft and resolving conflicts using speed control and routing constraints.
4. Learning-Based Optimization Layer
   Develop a reinforcement learning agent that proposes arrival management actions to improve traffic flow efficiency.
5. Hybrid Safety Architecture
   Implement a rule-based safety supervisor that verifies RL-proposed actions to guarantee separation constraints, route feasibility, and vertiport capacity limits.
6. Evaluation and Analysis
   Compare the hybrid approach with the rule-based baseline using metrics such as:

• Vertiport throughput
• Arrival delay
• Energy consumption
• Number of safety interventions
• Number of pad conflicts/intrusions

1. Reporting and Dissemination
   Document the research results in a thesis report. The report may optionally be structured as a conference-style research paper suitable for submission to an academic conference in air traffic management or UAM.

Expected Results

The thesis is expected to deliver:

• A BlueSky-based simulation framework for evaluating vertiport arrival management strategies. An existing framework can be used as a starting point.
• A prototype centralized hybrid arrival management architecture combining reinforcement learning with rule-based safety constraints.
• A comparative performance analysis between purely rule-based and hybrid traffic management approaches. The design of a purely rule-based arrival manager is available to use as a starting point.
• Insights into the potential advantages, limitations, and research challenges of applying learning-based optimization to UAM arrival management.

The results should contribute to improving understanding of how hybrid intelligent traffic management systems could support safe and efficient operations in future drone and Urban Air Mobility networks.

What do we expect from you?

• You are completing MSc in Aerospace Engineering, Artificial Intelligence, Robotics, Computer Science or comparable study at a Dutch university
• You have experience with programming in Python.
• You have experience with practical application of ML/RL (PyTorch, or others).
• You have completed ML/RL courses as part of your study
• Bonus: You have knowledge on Air Traffic Management

What we offer

• Enthusiastic colleagues who are experts in their field
• A flexible working space
• An environment where you have the opportunity to develop your skills and learn new ones
• A challenging assignment in a high-tech, result orientated work environment
• A thesis assignment allowance
• An informal corporate culture where your opinion counts!

About NLR

You will be working within the AOAP (Aerospace Operations: Air Traffic Management & Airports) department. Your colleagues are focused on solving real-world problems within air traffic management, airspace design, U-Space and other exciting domains.