Internship | Generative inpainting to create synthetic events for improved Vision-Language Models

About this internship

The ability to query visual data effectively is crucial for intelligence operations, yet current Vision-Language Models (VLMs) struggle with complex queries such as identifying ‘potential ambush situations’ in satellite images. Even more challenging are queries related to events, where images at multiple timepoints are needed to tell the full story. For example to find ‘suspicious behavior near airport fences’. Finetuning a VLM for queries related to such events would require many text-images combinations that illustrate such events, for which real-world examples are very hard to obtain. To overcome this lack of data, we focus on synthetic examples, making use of recent advances in generative AI, specifically diffusion-based image synthesis.

What will be your role?

Diffusion-based image synthesis methods offer an opportunity to create highly realistic image data, e.g. by inpainting objects. The goal of this student project is to explore novel diffusion methodology for inpainting events into sets of images. This requires strong control over the objects in the scene, and consistency between multiple images. After you have created the synthetic events, you will them to finetune VLMs and see how they can be improved to identify these events in real-world data.

You will perform this assignment within TNO’s Intelligent Imaging department. The Intelligent Imaging department is a passionate, creative, and dedicated team of professionals (60 people) specialized in developing groundbreaking applications in the field of computer vision. Intelligent Imaging is a young and growing department which has built up a lot of expertise over the past years in AI and deep learning.

What we expect from you

We are looking for a student that wants to join our cutting-edge generative AI research team to explore guided-diffusion techniques (or similar). This position is perfect for a student that is passionate about novel AI methods and defense applications. Additional requirements include that you are in the final stages of your master's degree in artificial intelligence, computer science, physics, mathematics, electrical engineering, or a similar degree. You have experience in computer vision, artificial intelligence, deep learning and Python-programming. The duration of the master project or internship is between 6 and 12 months.

What you'll get in return

You will, of course, be properly supervised during your work placement and be given the scope for you to get the best out of yourself. Furthermore, we provide:

• A highly professional, innovative internship environment, within a team of top experts.
• A suitable internship allowance (615 euro for wo-, hbo- and mbo-students, for a full-time internship).
• Possibility of eight hours of free leave per internship month (for a full-time internship).
• A free membership of Jong TNO, where you can meet other TNO professionals and join several activities, such as sports activities, (work-related) courses or the yearly ski-trip.
• Use of a laptop.
• An allowance for travel expenses in case you don’t receive an OV-card.

Security clearance requirement

For this internship vacancy it is required that the AIVD issues a security clearance (VGB) after conducting a security screening. Take into account that this process may take about 8 weeks. If you have been abroad for more than 6 consecutive months, or if you do not have the Dutch nationality, it may take longer.