Machine Learning Manager - GenAI Evaluation- Content Intelligence Team

Join Booking.com as a Machine Learning Manager - GenAI Evaluation on the Content Intelligence Team

About Us: At Booking.com, data drives our decisions. Technology is at our core. And innovation is everywhere. But our company is more than datasets, lines of code or A/B tests. We’re the thrill of the first night in a new place. The excitement of the next morning. The friends you encounter. The journeys you take. The sights you see. And the memories you make. Through our products, partners and people, we make it easier for everyone to experience the world.

Leadership/Team Quote:

The Content Intelligence track is responsible for developing machine learning solutions based on computer vision, natural language processing, and generative AI to drive supply enrichment and power a wide range of applications at Booking.com. This includes building GenAI agents for tasks such as dialog systems, Q&A, and trip planning, as well as developing foundational capabilities like agent infrastructure and evaluation. A key focus of the team is also the development of in-house travel-specific LLMs to support core use cases across the platform.

Role Description:

As a Machine Learning Manager, you will lead a team focused on the evaluation and optimization of generative AI systems. You will drive the development of Judge LLMs to assess model outputs across various tasks, oversee the design of robust evaluation frameworks for agentic workflows, and guide the creation of scalable pipelines for synthetic data generation. A key part of your role will be to ensure high-quality, multilingual evaluation that supports the global scalability of our GenAI applications. You will collaborate closely with cross-functional teams to align evaluation strategy with product goals and help shape the future of GenAI at Booking.com.

As a technical manager of Machine Learning Engineers and Machine Learning scientists, you should be passionate about technology, keep up to date with recent breakthroughs in the field, define and shape the team’s ML and platforms roadmap, and not be afraid to get your hands dirty with code when needed.

You are expected to be the focal point for all technical aspects, make sure your team members deliver on their tasks, and work together with other stakeholders to define and shape the roadmap of our products. You will work independently and will also be responsible for making technical decisions within your team.

When it comes to management, your expertise in handling people will motivate and inspire them to reach outstanding success! You should have experience in developing people. You will mentor and coach your team while working closely with a Product Manager.

Key Job Responsibilities and Duties:

• Lead and develop a high-performing team, fostering individual growth and collaboration.
• Manage and mentor ML engineers and ML scientists, ensuring their professional development and effectiveness.
• Develop scalable ML infrastructure and pipelines for efficient data processing and evaluations deployment.
• Evaluate architecture solutions based on cost, business needs, and emerging technologies.
• Contribute to generative AI development, including novel applications like GPT variants.
• Collaborate closely with software engineers to ensure seamless deployment and model inference.
• Monitor application health, set and track relevant metrics, and implement effective maintenance strategies.
• Collaborate with stakeholders to translate business requirements into viable ML solutions.
• Evaluate and integrate new ML technologies to enhance productivity and performance.
• Drive continuous improvement through model retraining, performance monitoring, and optimization.
• Develop robust ML and AI solutions that meet business objectives while considering production constraints.
• Stay abreast of industry methodologies, explore new technologies, and champion their adoption within the team.
• Actively contribute to Machine Learning at Booking.com through training, exploration of new technologies, and mentoring colleagues.
• Advocate for improvements, scaling, and extension of ML tooling and infrastructure.
• Foster a culture of innovation, collaboration, and excellence within the ML team.

Qualifications & Skills:

• 3+ years leading an ML team of a minimum of 4 people in a fast-paced production environment.
• Relevant work or academic experience (MSc + 5 years of working experience, or PhD + 3 years of working experience), involved in the application of Machine Learning to business problems.
• Masters degree, PhD or equivalent experience in a quantitative field (e.g. Computer Science, Engineering Mathematics, Artificial Intelligence, Physics, etc.).
• Advanced knowledge and experience in Computer Vision and Natural Language Processing, engineering aspects of developing ML and GenerativeAI models at scale.
• Experience designing and executing end-to-end solutions for deploying different ML models.
• Experience with cloud frameworks like AWS sagemaker for training, evaluation and serving models using TensorFlow, PyTorch, or scikit-learn.
• Experience with big data processing frameworks such, Pyspark, Apache Flink, Snowflake or similar frameworks.
• Demonstrable experience with MySQL, Cassandra, DynamoDB or similar relational/NoSQL database systems.
• Deep understanding of machine learning algorithms, statistical models, and data structures.
• Experience collaborating cross functionally in the development of machine learning products (e.g. Developers, UX specialists, Product Managers, etc.).
• Strong working knowledge of Python, Java, Kafka, Hadoop, SQL, and Spark or similar technologies. Working experience with version control systems.
• Excellent English communication skills, both written and verbal.
• Successfully driving technical, business and people related initiatives that improve productivity, performance and quality while communicating with stakeholders at all levels
• Leading by example, gaining respect through actions, not your title. Developing your team and motivating them to achieve their goals. Providing feedback timely and managing your key team performance indicators

Benefits & Perks - Global Impact, Personal Relevance:

• Annual paid time off and generous paid leave scheme including: parent, grandparent, bereavement, and care leave
• Hybrid working including flexible working arrangements, and up to 20 days per year working from abroad (home country)
• Industry leading product discounts - up to 1400 per year - for yourself, including automatic Genius Level 3 status and Booking.com wallet credit

Diversity, Equity and Inclusion (DEI) at Booking.com:

Diversity, Equity & Inclusion have been a core part of our company culture since day one. This ongoing journey starts with our very own employees, who represent over 140 nationalities and a wide range of ethnic and social backgrounds, genders and sexual orientations.

“At Booking.com, the diversity of our people doesn’t just build an outstanding workplace, it also creates a better and more inclusive travel experience for everyone. Inclusion is at the heart of everything we do. It’s a place where you can make your mark and have a real impact in travel and tech.”
Paulo Pisano, Chief People Officer

We ensure that colleagues with disabilities are provided the adjustments and tools they need to participate in the job application and interview process, to perform crucial job functions, and to receive other benefits and privileges of employment.

Booking.com is proud to be an equal opportunity workplace and is an affirmative action employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, gender, gender identity or expression, sexual orientation, national origin, genetics, disability, age, or veteran status. We strive to move well beyond traditional equal opportunity and work to create an environment that allows everyone to thrive.

Pre-Employment Screening

If your application is successful, your personal data may be used for a pre-employment screening check by a third party as permitted by applicable law. Depending on the vacancy and applicable law, a pre-employment screening may include employment history, education and other information (such as media information) that may be necessary for determining your qualifications and suitability for the position.