Data Engineer (SO2)

IT And Software Development National Capital Region Pasig City, National Capital Region

Knowledge, Skills & Experience Industry Experience

Pathways

DBMS administration and SQL development -> ETL specialist -> Cloud and Data Engineering.
Software Engineer specialist in back-office systems and data sets -> Data Engineering. Data Engineering Team lead experience is an advantage.

Software Engineering

A solid understanding of fundamental programming techniques and concepts such as:
object oriented programming patterns.
functional programming concepts.
distributed computing problems and solutions.
algorithm and data structure selection and optimization.
test-driven development.

A polyglot developer - experienced to expert, in at least two of the following languages:

Ideal

→ Go

→ Scala

→ Python

Advantageous

→ Java

→ C / C++

→ PHP

Data Engineering

General Data Architecture knowledge such as Data Lakes, Data Warehouses and related technologies and concepts.
Experience in, and an ability to express the comparative merits of SQL, NoSQL and Graph database systems.
Solid experience in at least two RDBMS’s with MySQL and PostgreSQL being highly preferred.
A high level of demonstrable SQL capability including query optimisation and tuning.
An understanding of the technology, use cases, and choices available, in the following technological groups:
Data Processing (e.g. Spark, Beam
Data Workflows (e.g. Airflow, Luigi)
Data Formats (e.g. Avro, Parquet, protobuf)
Messaging/Streaming Systems (e.g. Kafka, RabbitMQ, NATS)

DevOps and Cloud Engineering

Experience with infrastructure-as-code tooling: Ansible and Terraform.
Provisioning AWS services to support product deployment.
Metrics, monitoring, and alarms. Especially useful if you know Sumo Logic and PagerDuty.
AWS services related to data engineering

Advantageous Skills and Experience

Docker and related container-orchestration technologies (e.g. Kubernetes/AWS EKS).
Microservice architectures.
Data visualisation tools such as Tableau or Data Studio.
Finance industry domain knowledge, trading fundamentals and fintech systems (e.g. FIX protocol).