Senior Data Engineer
Senior Data Engineer
Shutterfly is looking for an experienced Data Engineer to join our growing team of data ninjas working on an analytics platform. You will be responsible for designing, expanding, and optimizing our data infrastructure and data pipeline architecture, as well as optimizing current data flows and data collection. You are an expert data integration and data infrastructure architect who enjoys optimizing data systems and building them from the ground up. You will support various data initiatives and will ensure optimal data delivery architecture throughout all ongoing projects. You must be self-directed and comfortable supporting the data needs of multiple teams, systems and products. You are also excited by the prospect of building from scratch, optimizing or re-designing our data architecture to support our next generation of products and initiatives.
The Expertise You Have:
• Bachelors or Master’s Degree in a technology related field (e.g. Engineering, Data Science, Computer Science)
• 10+ years of experience in multiple areas that include data architecture, data modeling, BI tools, data integration, agile software development
• Proven data architecture, data modeling, solution design, and data product design experience
• Proven track record of successful communication of data infrastructure, data models, and data engineering solutions through written communication, including an ability to effectively communicate with both business and technical teams
The Skills You Bring
• Develop and improve the current data architecture, data quality, monitoring and data availability.
• Building scalable data systems
• SQL, Python, Data Modeling, ETL/ELT/Real-time architectures, CDC/Event-driven architectures
• Help improve ongoing reporting and analysis processes, simplifying self-service support for customers
• Experience building and optimizing data pipelines and automation supporting data transformation, data structures, metadata, dependency, workload management, etc.
• Experience performing root cause analysis on data and processes to answer specific data/business questions
• Strong analytic skills working with structured and unstructured datasets
• Ability to perform in a fast-paced environment where continuous innovation is desired, and ambiguity is the norm
• Experience implement following software/tools/technologies:
o Message queuing, stream processing, and highly scalable ‘big data’ data stores
o Data pipelines for batches, queues, and streams: Spark, Kafka, Storm, Kinesis, etc.
o Databases such as MemSQL, MySQL, Postgres
o AWS cloud services: EC2, EMR, RDS, Redshift, SQS, Lambda
o Data pipeline and workflow management tools: Azkaban, Luigi, Airflow, etc.
o Object-oriented programming languages: Python, Rust, Scala, Java, etc.