PhD Position Acoustic Perception for Intelligent Vehicles

Passionate about AI-driven perception for intelligent vehicles? Join us to develop deep learning techniques for acoustic sensing. One PhD researcher (Topic A) will design new real-world ADAS applications based on sound detection, while another (Topic B) will focus on fusing acoustic data with other sensors for robust environment perception. Help shape the future of autonomous driving!

Job description

In this project you will develop novel AI and Deep Learning techniques for traffic detection using vehicle-mounted microphones, enhancing the perception capabilities of Advanced Driver-Assistance Systems (ADAS) and Automated Vehicles (AVs) beyond traditional camera, radar, and lidar sensors. Unlike these sensors, which require direct line-of-sight, acoustic detection captures traffic sounds like tire noise, sirens, and electric vehicle signals, enabling anticipation of unseen traffic in low-visibility scenarios. This additional sensing modality can improve driving safety, for instance by detecting approaching vehicles around corners or behind obstacles, and by localizing salient sounds such as sirens. Your research will explore new use cases of acoustic perception for advanced automated driving systems, and new methods for fusing sound with other sensor data for more robust environment perception through deep learning.

We offer two fully-funded 4-year PhD positions at the Intelligent Vehicles Group in the Cognitive Robotics department of TU Delft. The group frequently publishes in top conferences and journals such as CVPR, ECCV, ICRA, IROS, T-IV, T-ITS, T-RO, T-PAMI. Your primary supervisor will be Dr. Julian Kooij. The project is co-funded and co-supervised by the Audio Innovation Center of NXP Semiconductors in Leuven (NASDAQ: NXPI). For your work you will have access to the compute resources of TU Delft, ranging from personal machines, to shared GPU servers, the Delft AI Cluster that is shared across departments, as well as DelftBlue , which is one of the top 250 supercomputers in the world. The IV group’s Prius demonstrator vehicle with cameras, lidars, radars, and a roof-mounted microphone array will be used to investigate acoustic detection and localization salient traffic sounds. The project aims for viable real-time solutions and an online demonstration of acoustic ADAS.

Several challenges will be addressed, divided over the two PhD positions:

• Topic A) in this role, you will identify promising ADAS applications that can benefit from acoustic sensing and design novel use case-specific solutions with machine learning. You will build a working proof-of-concept system to be tested in controlled real-world experiments for at least one application. By aiming for minimal hardware and computational resources, the practical value for ADAS is maximized.
• Topic B) in this role, you will develop novel acoustic Deep Learning techniques to achieve state-of-the-art performance in multi-sensor environment perception for autonomous driving. You will develop models that provide holistic representations of all surrounding traffic by fusing acoustic with other sensor modalities. Such holistic traffic representations support many driving tasks at once, as required for full self-driving. By integrating acoustics you improve the robustness of the existing sensor suite.

At the Intelligent Vehicles Group you'll find an open and friendly environment, with opportunities for professional development and training to successfully develop your academic skills. We have a collaborative culture regarding research, education, and use of lab resources. We interact on a daily basis and share a drive to strengthen the position of the group as a whole thanks to the growth of each member.

Job requirements

• Completed (or about to complete) a MSc degree related to any of: artificial intelligence, machine learning, intelligent vehicles / robotics, acoustics and signal processing, computer vision.
• Good theoretic understanding of the fundamentals of machine learning.
• Ability to act independently as well as to collaborate effectively with members of a larger interdisciplinary team, take initiative, be result oriented, organized and creative.
• A certain affinity towards turning complex concepts into real-world practice (i.e. vehicle demonstrator) is desired.
• Excellent programming skills (Python or C/C++).
• Good verbal and written command of English.

Desirable for topic A:

• Hands-on experience with real-world data collection in robotics, intelligent vehicles, or a related area.
• Experience in applying Machine Learning / Deep Learning to real-world data.
• Knowledge of acoustic signal processing (filtering, beamforming, direction of arrival, SRP-PHAT) is a plus.
• Knowledge of the Robot Operating System (ROS) is a plus.

Desirable for topic B:

• Demonstratable experience in applying Deep Learning, using PyTorch, TensorFlow, JAX
• Experience with robotic/vehicle perception, preferably acoustics, but experience with computer vision, lidar, radar perception is also a plus.

TU Delft (Delft University of Technology)

Delft University of Technology is built on strong foundations. As creators of the world-famous Dutch waterworks and pioneers in biotech, TU Delft is a top international university combining science, engineering and design. It delivers world class results in education, research and innovation to address challenges in the areas of energy, climate, mobility, health and digital society. For generations, our engineers have proven to be entrepreneurial problem-solvers, both in business and in a social context.

At TU Delft we embrace diversity as one of our core values and we actively engage to be a university where you feel at home and can flourish. We value different perspectives and qualities. We believe this makes our work more innovative, the TU Delft community more vibrant and the world more just. Together, we imagine, invent and create solutions using technology to have a positive impact on a global scale. That is why we invite you to apply. Your application will receive fair consideration.

Challenge. Change. Impact!

Faculty Mechanical Engineering

From chip to ship. From machine to human being. From idea to solution. Driven by a deep-rooted desire to understand our environment and discover its underlying mechanisms, research and education at the ME faculty focusses on fundamental understanding, design, production including application and product improvement, materials, processes and (mechanical) systems.

ME is a dynamic and innovative faculty with high-tech lab facilities and international reach. It’s a large faculty but also versatile, so we can often make unique connections by combining different disciplines. This is reflected in ME’s outstanding, state-of-the-art education, which trains students to become responsible and socially engaged engineers and scientists. We translate our knowledge and insights into solutions to societal issues, contributing to a sustainable society and to the development of prosperity and well-being. That is what unites us in pioneering research, inspiring education and (inter)national cooperation.

Conditions of employment

Doctoral candidates will be offered a 4-year period of employment in principle, but in the form of 2 employment contracts. An initial 1,5 year contract with an official go/no go progress assessment within 15 months. Followed by an additional contract for the remaining 2,5 years assuming everything goes well and performance requirements are met.

Salary and benefits are in accordance with the Collective Labour Agreement for Dutch Universities, increasing from € 2872 per month in the first year to € 3670 in the fourth year. As a PhD candidate you will be enrolled in the TU Delft Graduate School. The TU Delft Graduate School provides an inspiring research environment with an excellent team of supervisors, academic staff and a mentor. The Doctoral Education Programme is aimed at developing your transferable, discipline-related and research skills.

The TU Delft offers a customisable compensation package, discounts on health insurance, and a monthly work costs contribution. Flexible work schedules can be arranged.

Will you need to relocate to the Netherlands for this job? TU Delft is committed to make your move as smooth as possible! The HR unit, Coming to Delft Service, offers information on their website to help you prepare your relocation. In addition, Coming to Delft Service organises events to help you settle in the Netherlands, and expand your (social) network in Delft. A Dual Career Programme is available, to support your accompanying partner with their job search in the Netherlands.