Data Engineer (SO2)

IT And Software Development National Capital Region Pasig City, National Capital Region


Description

Knowledge, Skills & Experience Industry Experience

 Pathways

  • DBMS administration and SQL development -> ETL specialist -> Cloud and Data Engineering.
  • Software Engineer specialist in back-office systems and data sets -> Data Engineering. Data Engineering Team lead experience is an advantage.

Software Engineering

  • A solid understanding of fundamental programming techniques and concepts such as:
  • object oriented programming patterns.
  • functional programming concepts.
  • distributed computing problems and solutions.
  • algorithm and data structure selection and optimization.
  • test-driven development.

 

  • A polyglot developer - experienced to expert, in at least two of the following languages:

Ideal

→ Go

→ Scala

→ Python

 

Advantageous

→ Java

→ C / C++

→ PHP

  • Comfortable working with Linux operating systems, scripts and packages.

Data Engineering

  • General Data Architecture knowledge such as Data Lakes, Data Warehouses and related technologies and concepts.
  • Experience in, and an ability to express the comparative merits of SQL, NoSQL and Graph database systems.
  • Solid experience in at least two RDBMS’s with MySQL and PostgreSQL being highly preferred.
  • A high level of demonstrable SQL capability including query optimisation and tuning.
  • An understanding of the technology, use cases, and choices available, in the following technological groups:
  • Data Processing (e.g. Spark, Beam
  • Data Workflows (e.g. Airflow, Luigi)
  • Data Formats (e.g. Avro, Parquet, protobuf)
  • Messaging/Streaming Systems (e.g. Kafka, RabbitMQ, NATS)

 

DevOps and Cloud Engineering

  • Experience with infrastructure-as-code tooling: Ansible and Terraform.
  • Provisioning AWS services to support product deployment.
  • Metrics, monitoring, and alarms. Especially useful if you know Sumo Logic and PagerDuty.
  • AWS services related to data engineering

Advantageous Skills and Experience

  • Docker and related container-orchestration technologies (e.g. Kubernetes/AWS EKS).
  • Microservice architectures.
  • Data visualisation tools such as Tableau or Data Studio.
  • Finance industry domain knowledge, trading fundamentals and fintech systems (e.g. FIX protocol).