Data Platform Engineer - OLAP

This is Adyen

Adyen provides payments, data, and financial products in a single solution for customers like Meta, Uber, H&M, and Microsoft - making us the financial technology platform of choice. At Adyen, everything we do is engineered for ambition.

For our teams, we create an environment with opportunities for our people to succeed, backed by the culture and support to ensure they are enabled to truly own their careers. We are motivated individuals who tackle unique technical challenges at scale and solve them as a team. Together, we deliver innovative and ethical solutions that help businesses achieve their ambitions faster.

Sitting at the intersection of Data Engineering, Backend Engineering, and Systems Engineering, Data Platform Engineers at Adyen build the foundational layer of tooling and processes for our on-premise Analytical Data Platforms. These tools support 10s of products, 100s of developers, and 1000s of daily jobs that add to Adyen’s strong portfolio of capabilities.

We’re looking for an expert with deep knowledge in distributed systems, to focus on our internal Online Analytical Processing (OLAP) ecosystem. You’ll collaborate with Data and ML Engineers to continuously improve this ecosystem. You’ll also collaborate with other platform engineers to position this ecosystem properly within the larger Data/AI/ML Platform capabilities, powered by Hadoop, Kubernetes, Spark, Trino, Flink, and Ray.

What you’ll do

Your responsibilities will focus on building new tools and improving existing tooling within the Data Platform ecosystem. Key focus areas include:

• Performance at Scale: Develop and maintain high-performance OLAP systems, supporting multi-tenant query workloads and ingestion pipelines with real Big Data scale.
• Reliability: Work with system reliability in mind, ensuring high availability for business-critical analytical products through observability and engineering excellence.
• Productize the Platform: Build self-service tooling that enables Data Engineers and Analysts to independently manage their data assets and diagnose issues.
• Data Quality: Engineer automated frameworks to validate data integrity and leverage metadata-driven tools to enhance data discoverability, lineage, and cataloging across the ecosystem.
• Ecosystem Integration: Architect seamless integrations of the OLAP ecosystem with adjacent distributed systems (e.g. storage, messaging, and batch / stream processing systems)
• Efficiency & Governance: Monitor and optimize cluster resource-efficiency while making sure the platform adheres to global security and data privacy standards.

Who you are

• Fluency in Python and/or Java. Golang or Rust are also appreciated.
• Team player with strong communication skills
• Ability to work closely with diverse stakeholders you enable (analysts, data scientists, data engineers, etc.) and depend upon (infrastructure, security, etc).
• Experience in OLAP technologies, like Druid, Clickhouse, Pinot, Doris, Starrocks, etc
• Experience in CI/CD pipelines, for code and infrastructure automation.
• Experience in Kubernetes
• Experience in infrastructure and large-scale private cloud systems
• Additional experience developing and maintaining:

• Other distributed data and compute systems like Spark, Trino, etc
• Data modelling for databases
• Real-time and batch data pipelines (via Kafka, Spark streaming) with an eye for frameworks, and emphasis on user friendliness and quality..