Data Engineering Lead (SO2)
Description
KEY RESPONSIBILITIES
- Lead the development and continuous improvement of the data engineering framework, standards, practices, and procedures
- Act as the Design Authority for all Data Engineering solutions
- Plan, prioritise and execute data engineering related activities to support the Data and Analytics Strategy
- Perform an ongoing maturity assessment of data engineering practices and identification of any people, process & technology needs to close any gaps
- Lead the design, deployment of management of the data lake and data warehouse capabilities, and the orchestration of data acquisition/ingestion and integration activities
- Develop and implement best practices in metadata management, data modelling and DevOps
- Engage and communicate with key business stakeholders such as product owners, data analysts etc and drive sprint planning to ensure the continuous delivery of business value
- Grow and mentor a team of data engineering specialists
ESSENTIAL REQUIREMENTS
- A tertiary qualification in Information Technology is highly regarded
- A minimum of 5 years’ experience in similar role with demonstrated experience leading data engineering related initiatives
- Highly proficient and extensive experience in the use of data engineering technologies such as Azure Data Factory, Databricks etc
- Highly proficient and extensive experience acquiring/ingesting data from structured/ununstructured, API’s and IoT related data sources
- Experience working with a Lambda data architecture and data modelling methodologies such as Kimball, Inmon and Data Vault
- Experience working with, and knowledge of, data governance, data quality, reference and master data management, and business glossary technologies
- Highly proficient and extensive experience in the use of programming languages such as T-SQL, JavaScript, Python, PowerShell etc
- Highly proficient and extensive experience in the use of data storage formats such as Parquet, Delta, ORC, JSON etc
- Excellent interpersonal and both oral and written communication skills
- Ability to build relationships with a variety of stakeholders
- Experience managing/leading a team
- Capacity to align strategic and operational needs with organisational development initiatives
- Solid project management and agile development experience
- Ability to travel