DATA PLATFORM ENGINEER - LONDON (HYBRID WORKING) - £75K + BENEFITS

We are looking for a Data Platform Engineer to join the Core Engineering team, where you'll architect the core platform in which the development teams execute highly complex data pipelines. You'll design and build solutions that scale compute to millions of concurrent job executions, optimised to meet performance, efficiency and cost-effectiveness requirements, while ensuring high availability.

Data Platform Engineers bridge the gap between software and infrastructure and are essential to the success of the companies technology strategy.

Responsibilities:

• Design and build software solutions to scale AWS compute resources to meet application performance requirements.
• Ensure 24/7 system reliability by implementing company and industry best practices in replication, redundancy and monitoring.
• Implement workflow management software, to automate operational tasks and optimise the utilisation of infrastructure and applications.
• Design and implement CI/CD workflows to maintain software quality via continuous and automated deployment and testing.
• Work with development and operations teams to design solutions to complex problems, involving large data pipelines that process terabytes of historical market data, in the most efficient and cost-effective manner.
• Regularly review and assess new tools that become available in the industry and assess how they could be integrated into the platform to continuously improve.

Requirements:

Essential

• Industry experience with cloud computing tools and services in complex systems, preferably in AWS.
• Strong Python programming skills.
• Industry experience with software development lifecycle processes and tools.
• Experience working in a Linux environment.
• Experience with Docker.
• Experience with SQL and relational databases.
• Avid learner, problem solver and detail-orientated.
• Excellent teamwork and the ability to communicate and work in multidisciplinary teams in a collaborative manner.
• Computer science or other STEM degree.
• At least two years of industry experience.

Desirable

• Familiarity with distributed systems concepts and tools, such as Spark, Ray, RabbitMQ, Kafka, AWS Batch.
• Familiarity with DevOps practices and tools, such as Terraform.
• Familiarity with job execution and orchestration tools, such as Celery and Airflow.

This is a great opportunity to join an exciting company going through a period of growth, with opportunities for career progression. For more information and a full job description, please contact Ben Turner at BRT Consulting.