Internship | From digital to physical: can adversarial patches fool AI?

About this internship

The robustness of object detection models is crucial for them to be usable in a military context. Adversarial patches, designed to mislead models, are a threat to this robustness. These patches are usually generated and evaluated in the digital domain and using bright colors. In this research project, you will investigate if these patches work in the real world. You will create adversarial patches, create a copy of them in the real world and make recordings of these patches to validate how effective they are in the real world. Your work will be crucial in assessing the threat of adversarial AI to the Dutch Ministry of Defense.

What will be your role?

You will study different methods to generate adversarial patches that can be used to hide objects from object detection models (e.g. YOLO, RT-DETR). You will optimize these patches for one or more models and find a way to transfer these digital patches to the physical domain (e.g. by printing them onto a board or creating a T-shirt). You will then investigate how well these physical patches work compared to their digital counterparts. One crucial aspect that you could evaluate is how distance between patch and camera influences the effectiveness of the patch to hide the object from detection. Another interesting aspect is balancing the stealthiness of the patch to both a human observer and an AI. You could also consider varying the angles you are recording from, the lighting conditions, the method of creating the physical patch, and the model the patch is optimized and evaluated on.

You will perform this assignment within TNO’s Intelligent Imaging department. The Intelligent Imaging department is a passionate, creative, and dedicated team of professionals (60 people) specialized in developing groundbreaking applications in the field of computer vision. Our team members have diverse backgrounds, ranging from the medical field to artificial intelligence. Intelligent Imaging is a young and growing department which has built up a lot of expertise over the past years in AI and deep learning.

What we expect from you

You are in the final stages of your master's degree in artificial intelligence, computer science, physics, mathematics, electrical engineering, or a similar degree. You have some experience in computer vision, artificial intelligence, specifically deep learning and programming in Python. In addition to technical experience, we value clear communication, a result-driven attitude, and a motivation to publish the work.

What you'll get in return

You want an internship opportunity on the precursor of your career; an internship gives you an opportunity to take a good look at your prospective future employer. TNO goes a step further. It’s not just looking that interests us; you and your knowledge are essential to our innovation. That’s why we attach a great deal of value to your personal and professional development. You will, of course, be properly supervised during your work placement and be given the scope for you to get the best out of yourself. Furthermore, we provide:

• A highly professional, innovative internship environment, within a team of top experts.
• A suitable internship allowance (615 euro for wo-, hbo- and mbo-students, for a full-time internship).
• Possibility of eight hours of free leave per internship month (for a full-time internship).
• A free membership of Jong TNO, where you can meet other TNO professionals and join several activities, such as sports activities, (work-related) courses or the yearly ski-trip.
• Use of a laptop.
• An allowance for travel expenses in case you don’t receive an OV-card.

TNO as an employer

At TNO, we innovate for a healthier, safer and more sustainable life. And for a strong economy. We find each other in wonder and ingenuity. We are driven to push boundaries. There is all the space and support for your talent and ambition. You work with people who will challenge you: who inspire you and want to learn from you. Our state-of-the-art facilities are there to realize your vision. What you do at TNO matters: impact makes the difference. Because with every innovation you contribute to tomorrow’s world.