Data Engineer (SO2)
Description
KEY RESPONSIBILITIES
- Contribute towards the development and continuous improvement of the data engineering framework, standards, practices, and procedures
- Contribute towards data-driven initiatives to support the Data and Analytics Strategy
- Design, build, test, deploy and maintain data engineering solutions
- Scope, estimate, prioritise and execute data analytics related activities and work collaboratively with the Data Management and Data Analytics teams
- Contribute towards an ongoing maturity assessment of data analytics practices
- Contribute towards the design, deployment of management of the data lake and data warehouse capabilities, and orchestration of data acquisition/ingestion and integration activities
- Contribute towards the development, adoption, and ongoing management of metadata management, data modelling and DevOps best practices
- Engage and communicate with key business stakeholders such as product owners, data analysts in support of the continuous delivery of business value
Essential requirements
- A tertiary qualification in Information Technology is highly regarded
- A minimum of 3 years’ experience in similar role with demonstrated experience implementing data engineering solutions
- Highly proficient in the use of data engineering technologies such as Azure Data Factory, Databricks etc
- Highly proficient in acquiring/ingesting data from structured/un-unstructured, API’s and IoT related data sources
- Experience working with a Lambda data architecture and data modelling methodologies such as Kimball, Inmon, and Data Vault
- Experience working with, and knowledge of, data governance, data quality, reference and master data management, and business glossary technologies
- Highly proficient and extensive experience in the use of programming languages such as T-SQL, JavaScript, Python, PowerShell etc
- Highly proficient in the use of data storage formats such as Parquet, Delta, ORC, JSON etc
- Excellent interpersonal and both oral and written communication skills
- Ability to build relationships with a variety of stakeholders
- Sound experience in Agile ways of working
- Ability to travel
Desirable
- Industry knowledge of agriculture, food, or supply chain
- Have acquired a Data Engineering certification from a vendor such as Microsoft, Databricks etc.