Senior Data Engineer

About the Role:

We are currently seeking a Senior Data Engineer to join our team to develop and maintain scalable data pipelines to support growing data volume and complexity. This position involves close interaction with cross-functional teams to improve data models that feed business intelligence tools, increasing data accessibility and fostering data-driven decision making across the organization.

Responsibilities:

Data Pipeline Development and Maintenance:

• Design, build, and maintain data pipelines and ETL processes on AWS to support analytics and machine learning initiatives.

• Use AWS services such as RDS, Glue, Lambda, S3 to manage and process large data sets.

DevOps and Automation

• Incorporate DevOps best practices into data engineering workflows, focusing on automation, CI/CD, and infrastructure as code (IaC) for repeatable, scalable solutions.

• Develop and maintain IaC templates (e.g. Terraform, CloudFormation) to provision and manage AWS resources for data engineering and ML tasks.

• Build monitoring, logging, and alerting systems to ensure data pipeline and model uptime, performance, and data quality.

Qualifications:

• Must hold hands-on experience with ETL process with at least 5+ years of experience.

Knowledge, Skills and Abilities Required:

• Cloud Expertise: Strong experience with AWS, including services like S3, Glue, Lambda, and SageMaker.

• Programming: Advanced proficiency in Python, including experience with data processing libraries (e.g. Pandas) and automation.

• DevOps: Experience with DevOps practices, including CI/CD pipelines, Infrastructure as Code (Terraform, CloudFormation), and monitoring (CloudWatch).

• Data Engineering: Solid understanding of data warehousing, ETL/ELT processes.

• Collaboration: Strong communication skills and experience working in cross-functional Agile teams.

• Excellent analytical and problem-solving skills.

• Effective listening skills to understand the requirements of the business.

• Planning, time management and organisational skills.

• The ability to deliver under pressure and to tight deadlines.