Sr. Data Engineer
Description
Sr. Data Engineer
Job Overview:
We are seeking an experienced Technical Data Engineer / ELT Developer with expertise in data integration and pipeline development using Snowflake. The ideal candidate will manage data ingestion and transformation, working with an offshore team to deliver efficient and scalable data solutions that align with business requirements and performance standards.
We are seeking an experienced Technical Data Engineer / ELT Developer with expertise in data integration and pipeline development using Snowflake. The ideal candidate will manage data ingestion and transformation, working with an offshore team to deliver efficient and scalable data solutions that align with business requirements and performance standards.
Key Responsibilities:
- Data Pipeline Development: Design, build, and maintain robust ELT (Extract, Load, Transform) pipelines using Snowflake to support data ingestion, integration, and transformation.
- Technical Leadership: Lead offshore development teams in implementing best practices for data engineering and ELT development.
- Data Integration: Collaborate with stakeholders to understand data sources and integration requirements, ensuring seamless connectivity and data flow between systems.
- Performance Optimization: Optimize data pipelines for performance, scalability, and reliability, including query tuning and resource management within Snowflake.
- Data Quality Assurance: Implement and monitor data validation procedures to ensure data accuracy and consistency across systems.
- Collaboration and Communication: Work closely with project managers, data architects, and business analysts to align project milestones and deliverables with business goals.
- Documentation: Create and maintain detailed documentation of data pipelines, data flow diagrams, and transformation logic.
- Issue Resolution: Troubleshoot and resolve issues related to data pipelines, including job failures and performance bottlenecks.
- Compliance and Security: Ensure all data management processes comply with data governance policies and regulatory requirements in financial services.
Required Qualifications:
- Bachelor’s degree in Computer Science, Information Technology, or a related field.
- 5+ years of experience in data engineering with a strong focus on ELT processes and data pipeline development.
- Hands-on experience with Snowflake cloud data platform, including data sharing, secure views, and performance optimization.
- Proficiency in SQL and familiarity with data integration and ETL/ELT tools.
- Experience managing and collaborating with offshore development teams.
- Strong problem-solving skills and the ability to work independently to meet deadlines.
- Excellent communication skills for effectively interacting with technical and non-technical stakeholders.
Preferred Qualifications:
- Certifications in Snowflake or relevant data technologies.
- Experience in the financial services sector with an understanding of data security and compliance requirements.
- Familiarity with cloud platforms (e.g., AWS, Azure) and data orchestration tools (e.g., Apache Airflow).
- Experience with scripting languages such as Python or JavaScript for data transformation.
- Knowledge of data visualization tools (e.g., Tableau, Power BI).