Staff Software Engineer - Observability

This is Adyen

Adyen provides payments, data, and financial products in a single solution for customers like Meta, Uber, H&M, and Microsoft - making us the financial technology platform of choice.

For our teams, we create an environment with opportunities for our people to succeed, backed by the culture and support to ensure they are enabled to truly own their careers. We are motivated individuals who tackle unique technical challenges at scale and solve them as a team. Together, we deliver innovative and ethical solutions that help businesses achieve their ambitions faster.

Staff Software Engineer - Platform Observability

Adyen provides a global, unified platform with a wide variety of financial services. Observability plays a key role providing comprehensive data to a wide variety of use cases, ensuring the platform’s reliability and supporting our operations to run smoothly.

As part of the Observability team you will build and maintain products and services that enable engineers at Adyen to understand how their services are behaving in real-time, reliably diagnose issues and streamline data discovery through a common observability ecosystem. Between logs, metrics and traces, our platform collects and processes billions of events per day.

What you will do:

As a Staff Software Engineer working on the observability of the platform, you will play a key role in shaping how teams work with telemetry data, driving strategic decisions, identifying and advocating for best practices for building, running and maintaining observable components.

This includes:

• Define and lead Logging, Metrics, Tracing & Alerting strategies
• Solve scaling bottlenecks in critical services in our telemetry data pipelines
• Create advanced tooling to accelerate root-cause analysis, reduce Mean Time to Resolution (MTTR), and eliminate alert fatigue
• Guide and mentor software engineering and SRE teams on monitoring best practices and instrumenting code.
• Identify and tackle bottlenecks and single points of failures in the architecture aiming to improve the reliability of the observability platform and the overall user experience.
• Driving features and products idealization, implementation and adoption.
• Collaborate on planning and refinement proactively providing input whilst striving for engineering alignment, quality and reducing complexity and dependencies.
• Respond to alerts and duty responsibilities, like providing support to other engineers and troubleshooting production issues

What you bring:

• You have 5+ years of relevant work experience with highly distributed systems
• Expertise in designing and implementing APIs and data pipelines for high-throughput, real-time data ingestion
• Experience improving software reliability between different categories including availability, performance, latency, efficiency, capacity, SLOs and incident management.
• Proficiency developing and maintaining software and frameworks written in Go and/or Java
• Advanced understanding of system design and evaluating its tradeoffs
• Strong stakeholder management, technical communication, and mentorship capability.
• Highly motivated to learn and continuously develop yourself
• Experience with containerisation and orchestration technologies (Docker, Kubernetes) and infrastructure as code tools(e.g: Terraform)

Observability Stack Expertise

You have hands-on experience operating core telemetry data stores at scale e.g. Elasticsearch/Opensearch/VictoriaLogs/Clickhouse for logging, Prometheus/VictoriaMetrics for metrics and Grafana Tempo for distributed tracing, Grafana LGTM stack, OpenTelemetry, Alertmanager, Clickhouse

Nice to haves:

• Experience with highly available/fault tolerant, replicated data storage systems, large scale data processing systems is a strong plus
• Infrastructure and Platform Experience
• Contributions to open-source observability projects