Data Engineer
AWS Cloud Data Engineer
Azure Databricks and PySpark experience is must
AWS Solution architect
- Must be Certified Solution Architect
- Familiar with
- AWS Glue
- EMR
- Kinesis
- S3
- Kubernetes/Docker.
Project Description:
Support one of the top Australian banks as they seek to modernise their data and analytics platform.
You will be working directly with IT and business stakeholders in Data and Platform team to implement banks data strategy to become the best AI bank of the world.
Responsibilities:
– Create technology blueprints and engineering roadmaps for multi-year data transformational programs.
knowledgeable of architectural patterns, cloud-native solutions on AWS, microservices architecture, solutions integration and containerisation.
– Extensive knowledge of AWS services, particularly those related to data storage and processing, including S3, RDS, Redshift, DataZone and Glue.
– Expert in the data development lifecycle, with a focus on data ingestion processes, data transformation pipelines, data integration, and visualisation.
– Excellent stakeholder management skills and ‘Can-do’ attitude
Mandatory Skills Description:
– 8+ years of hands-on experience with building modern data technology stacks and tooling, including database experience with Snowflake, Redshift and Aurora
– AWS Solution Architect certification
– Hands on experience in AWS compute and services including Redshift & EMR serverless, Airflow, Glue, Snowflake, Redshift, Aurora PostgreSQL
– Worked on event driven architecture using Apache Kafka, AWS Kinesis, or similar Infrastructure as code using CloudFormation or Terraform
– Proficient in Python, SQL
– Building CI/CD tooling (Github/Jenkins)
– Experience with Parquet File Format, Iceberg tables
– Containerisation (Docker, Kubernetes)
– Ab initio or DBT tooling
– Alation, Glue Data Catalogue & AWS DataZone
– Data visualisation tools and integration to Tableau, PowerBI
– Observability tools (i.e. Observe, Splunk or Prometheus/Grafana).
- Nice-to-Have Skills Description:
Markets domain knowledge particularly in dealing with Murex/MRE/Wallstreet datasets
