Internship | Infrared Foundation Model

Internship | Infrared Foundation Model

What will be your role?

Introduction

Foundation models have changed computer vision. Due to their broad pre-training, they are ideal starting points for further finetuning for a range of downstream tasks, leading to new possibilities even with limited amounts of training data. For RGB image tasks, foundation models have become a default choice. However, for infrared images this is not so straightforward. Recently, a foundation model for infrared images was proposed (Liu et al, ECCV 2024), InfMAE, using an information-aware masking strategy, because infrared images exhibit less diversity and less information content (e.g. less texture).

Surprisingly, finetuning this infrared model for object detection and semantic segmentation, did not result in significantly better results, compared to finetuning a large RGB model. Both for the finetuned RGB models as well as the infrared model, a good performance could be achieved on the infrared tasks. Sometimes a RGB model performed even better than the infrared model. That begs the question: how to get most out of RGB foundation models (e.g. DINO v2) for infrared tasks?

What will you be doing?

Explore the opportunities of foundation models for infrared images. Research how to transfer the knowledge in RGB to infrared and how to finetune it adequately for infrared. Investigate the added value for object detection by integrating the recipes into a modern model such as Grounding DINO.

You will perform this assignment within TNO’s Intelligent Imaging department. The Intelligent Imaging department is a passionate, creative, and dedicated team of professionals (60 people) specializing in developing groundbreaking applications in the field of computer vision. Our team members have diverse backgrounds, ranging from the medical field to artificial intelligence. Intelligent Imaging is a young and growing department that has built up a lot of expertise over the past years in AI and deep learning.

What we expect from you

We are looking for a student that wants to join our cutting-edge research team. This position is perfect for students who are passionate about AI and advanced machine learning techniques. Additional requirements include that you are in the final stages of your master's degree in artificial intelligence, computer science, physics, mathematics, electrical engineering, or a similar degree. You have some experience in computer vision, artificial intelligence, deep learning and Python-programming.

Please include the following practical information in your cover letter:

• Whether you are looking for an internship or a graduation project;
• The preferred duration of your project (flexible in the range of 6 to 12 months);
• Your desired start date (flexible, but due to the required security screening the earliest possible start date is three months after submitting your application).

What you'll get in return

You want an internship opportunity on the precursor of your career; an internship gives you an opportunity to take a good look at your prospective future employer. TNO goes a step further. It’s not just looking that interests us; you and your knowledge are essential to our innovation. That’s why we attach a great deal of value to your personal and professional development. You will, of course, be properly supervised during your work placement and be given the scope for you to get the best out of yourself. Furthermore, we provide:

• A highly professional, innovative internship environment, within a team of top experts.
• A suitable internship allowance (615 euro for wo-, hbo- and mbo-students, for a full-time internship).
• Possibility of eight hours of free leave per internship month (for a full-time internship).
• A free membership of Jong TNO, where you can meet other TNO professionals and join several activities, such as sports activities, (work-related) courses or the yearly ski-trip.
• Use of a laptop.
• An allowance for travel expenses in case you don’t receive an OV-card.

TNO as an employer

At TNO, we innovate for a healthier, safer and more sustainable life. And for a strong economy. Since 1932, we have been making knowledge and technology available for the common good. We find each other in wonder and ingenuity. We are driven to push boundaries. There is all the space and support for your talent and ambition. You work with people who will challenge you: who inspire you and want to learn from you. Our state-of-the-art facilities are there to realize your vision. What you do at TNO matters: impact makes the difference. Because with every innovation you contribute to tomorrow’s world.

At TNO we encourage an inclusive work environment, where you can be yourself. Whatever your story and whatever unique qualities you bring to the table. It is by combining our unique strengths and perspectives that we are able to develop innovations that make a real difference in society.