Internship | Reliable Image Classification with Vision-Language Models

About this position

Modern image classification has made tremendous progress thanks to big data. Deep learning models like CLIP, trained on over 400 million images, can now easily distinguish between different cat species and even military vehicles with impressive accuracy. This works well when we have high-quality photos. However, things become much less reliable when objects must be classified in less-than-ideal photos. Occlusion, noise, and low resolution significantly degrade classification performance. To estimate whether classifications made by these models are reliable, we can use their internal confidence scores. However, this is only a single signal and does not always reflect true reliability. CLIP and newer VLMs are increasingly used to score “look & feel” or image quality in a mostly training-free or lightly supervised way. These quality signals work surprisingly well, but they are usually scalar and not calibrated for decision-making.

What will be your role?

In defence, safety, and security applications, where data are scarce and conditions are harsh (occlusion, blur, low light, clutter), we don’t just need a label, we need to know when not to trust it. This thesis focuses on exactly that: can we turn extra image condition signals produced by CLIP or other VLMs into calibrated reliability estimates for image classification methods? Your goal is to design methods that predict whether a decision is reliable, improving end-to-end performance in fine-grained, low-data regimes.

During this internship, you will:

• Investigate novel approaches to better estimate the quality of image classifications.
• Study the current state-of-the-art in quality assessment of images themselves, the reliability of classification models, and possible methods for estimating auxiliary attributes of images.
• Explore new methods for quality assessment of images and their classifications, and compare them in defense scenarios to determine if these methods improve the reliability of classification.
• Optionally, write a scientific paper on your findings.

Your research could make a direct impact on object recognition systems for defense, safety, and security, and potentially improve accuracy and reliability in critical identification tasks. The proposed solutions could enhance the capability of AI systems to leverage domain expertise without requiring extensive image datasets.

You will perform this assignment within TNO’s Intelligent Imaging department. The Intelligent Imaging department is a passionate, creative, and dedicated team of professionals (60 people) specialized in developing groundbreaking applications in the field of computer vision. Our team members have diverse backgrounds, ranging from the medical field to artificial intelligence. Intelligent Imaging is a young and growing department which has built up a lot of expertise over the past years in AI and deep learning.

What we expect from you

We are looking for a highly motivated student to join our cutting-edge research team working on computer vision. You should be passionate about AI, and bridging computer vision and natural language processing. Furthermore, you are in the final stages of your master's degree in artificial intelligence, computer science, mathematics, (bio)medical engineering, or a related field. You have experience with computer programming (e.g. Python) and deep learning frameworks (e.g. PyTorch). Prior experience with computer vision and natural language processing concepts, and familiarity with transformer architectures and vision-language models is appreciated.

Your supervisor is an experienced TNO-colleague who is actively involved in ongoing research projects at TNO. He has experience both in state-of-the-art image analysis research, as well as supervising MSc students.

Please include the following practical information in your cover letter:

• Whether you are looking for an internship or a graduation project,
• The preferred duration of your project (flexible in the range of 6 to 12 months),
• Your desired start date (flexible, but due to the required security screening the earliest possible start date is three months after submitting your application).

What you'll get in return

You want an internship opportunity on the precursor of your career; an internship gives you an opportunity to take a good look at your prospective future employer. TNO goes a step further. It’s not just looking that interests us; you and your knowledge are essential to our innovation. That’s why we attach a great deal of value to your personal and professional development. You will, of course, be properly supervised during your work placement and be given the scope for you to get the best out of yourself. Furthermore, we provide:

• A highly professional, innovative internship environment, within a team of top experts.
• A suitable internship allowance (615 euro for wo-, hbo- and mbo-students, for a full-time internship).
• Possibility of eight hours of free leave per internship month (for a full-time internship).
• A free membership of Jong TNO, where you can meet other TNO professionals and join several activities, such as sports activities, (work-related) courses or the yearly ski-trip.
• Use of a laptop.
• An allowance for travel expenses in case you don’t receive an OV-card.

TNO as an employer

At TNO, we innovate for a healthier, safer and more sustainable life. And for a strong economy. Since 1932, we have been making knowledge and technology available for the common good. We find each other in wonder and ingenuity. We are driven to push boundaries. There is all the space and support for your talent and ambition. You work with people who will challenge you: who inspire you and want to learn from you. Our state-of-the-art facilities are there to realize your vision. What you do at TNO matters: impact makes the difference. Because with every innovation you contribute to tomorrow’s world.

Contact

E-mailadres: muriel.vanderspek@tno.nl