Sr. Data Engineer, AWS Platform and Infrastructure
Shutterfly is seeking an experienced Data Engineer with software engineering skills to join the Data Warehouse, Platform and Infrastructure team. You will own, manage and drive end-to-end solutions in data infrastructure and platform operations.
As part of the Platform and Infrastructure team, you will tackle the scalability, operations, security, performance, governance and compliance challenges of the Data Warehouse that collect, process and store data for a $2B customer eCommerce, photo, and manufacturing business. You will be part of building next generation Data Warehouse platform at Shutterfly on AWS, supporting the breadth and depth of the company’s analytic needs, including BI, Data Scientists, Product Management, Product Marketing, CRM, as well the Machine Learning teams providing innovations in our Websites, Mobile Apps and Manufacturing Labs.
The Data Warehouse, Platform and Infrastructure vision is to provide Shutterfly team’s the ability to manage the full life cycle of their data at all levels; simplifying, commoditizing and democratizing its collection, computation and analytics through well-architected use of the AWS.
We are looking for candidates who have expertise in building distributed and high-performance systems on AWS with sound data operational, infrastructure and continuous integration/deployment engineering skills.
* Design, build and maintain data warehouse infrastructure on AWS environment with automation tools such as Cloud Formation, Terraform, Code pipeline, Jenkins.. etc
* Lead automation projects for meeting and exceeding data availability SLAs, high data quality and infrastructure uptime and readiness. Build continuous integration/deployment (CI/CD) pipelines to accelerate development and improve team agility
* Monitor & manage all aspects of system security, performance, storage, incidents, usage, backup and recovery – for databases, applications, and infrastructure on AWS
* Maintain and support Redshift Cluster and Schemas and business pipelines. Take ownership of the design, governance, performance tuning, capacity planning, data availability and operational aspects of the data solutions
* Use your coding skills across a number of languages (Python and/or Java) and develop appropriate instrumentation to collect metrics on system performance, cost, data ingress/egress /storage processes
* Work with information security and compliance teams in governing policies and procedures
* Have a clear understanding of the reports/analyses/insights to be driven by data and build data driven solutions to optimally support the operational analytics needs
* Coordinate infrastructure enhancements and maintenance with the system/network engineering teams
* Perform technology evaluations and testing to introduce new technologies to be adopted by the data warehouse team
* Participate in 24x7 on-call rotation to handle issues that occur outside of business hours
* At least 5 plus years of experience with building and maintaining distributed and high-performance production environments in large-scale consumer enterprises.
* At least 1+ years of development and deployment experience with most if not all of the following components of AWS:
* Management - CloudWatch (Events/Logs), IAM, CloudTrail, EC2 Systems Manager
* IaaS - EC2, VPC, EBS, ELB, KMS, Config, SNS, SQS, SES, SWF, S3, Glacier
* Data Management - Redshift, RDS, EMR, Data Pipeline, Kinesis, DynamoDB
* Other - Server Migration, Storage Gateway, CDN
* Hands on experience with development and deployment of infrastructure with automation tools such as Cloud Formation, Terraform…. etc.
* Experience in designing and developing CI/CD pipelines with AWS code pipeline, code build, lambda, .. etc
* Strong CS fundamentals, experience with one of the programming languages (Java, Python) and able to write pipelines for automation, metrics/data collection and system administration purposes
* Understanding of automation and orchestration platforms such as Airflow, .. etc
* Ability to learn quickly and think outside of the box with excellent communication skills (written and oral)
* Bachelor’s degree in Computer Science or equivalent
* AWS Associate Level Certification required (Professional desired)