Data Engineer

The Wow

This role sits at the heart of our real-time data platform, empowering teams to build, deploy, and scale data products that drive critical business decisions and customer experiences.

You will design, develop, and operate the core platform capabilities that power streaming data pipelines, event-driven applications, and self-service data products. Working at the intersection of software engineering and data engineering, you will create scalable frameworks, tooling, and developer experiences that enable our teams to deliver reliable solutions with speed and confidence.

A taster of what you will be involved with: 

• Design and build scalable, reusable frameworks and services for data ingestion, transformation, and processing.
• Develop and maintain the platform's core capabilities, such as orchestration, data quality validation, and schema management tools.
• Evolve the platform's architecture to incorporate new technologies that improve scalability, reliability, and cost-effectiveness.
• Collaborate with Data DevOps Engineers to ensure platform features are automated, observable, and deployed via CI/CD.
• Act as a key contributor to the platform's roadmap, identifying and prioritizing features that unlock the most value for its users.

What we are looking for:

Must haves: 

• University Degree in Computer Science, Engineering or equivalent.
• At least 3 years of hands-on experience building and operating real-time or event-driven data platforms and streaming applications.
• Strong software engineering fundamentals, with experience in a language like Python, Scala or Java.
• Experience building reusable frameworks, libraries, or internal tools.
• Deep knowledge of SQL and data integration patterns. Experience in modern cloud database technologies such as Snowflake or Redshift are considered an asset.
• Experience with data streaming technologies such as Kafka is a must.
• Strong experience building distributed data processing workloads using Apache Spark (PySpark and/or Scala).
• Hands-on experience designing and operating event-driven architectures on AWS, including services such as MSK, Kinesis Data Streams, EKS, EMR, Lambda, and EventBridge.
• Experience in NoSQL databases such as  HBase, Cassandra, and DynamoDB.
• Excellent verbal and writing English communication skills.

Nice to have: 

• Familiarity with real time reporting tools like Grafana or Datadog