Sr Data Engineer

Product & Engineering Boston, MA United States


Description

Company Overview 

AtScale enables smarter decision-making by accelerating the flow of data-driven insights. The company's semantic layer platform simplifies, accelerates, and extends business intelligence and data science capabilities for enterprise customers across all industries. With AtScale customers are empowered to democratize data, implement self-service BI and build a more agile analytics infrastructure for better more impactful decision making.  

Job Description


Headquartered in Boston, we are hiring a Sr. Data Engineer to collaborate on cutting edge solutions for complex data operations in support of AI workflows. This is an opportunity to help build a new business from the ground up, within an existing market and company. 

 
Responsibilities
  • Work with data scientists to support data operations to support AI
  • Collaborate with developers and product managers to research, design, and implement integrations between AtScale technologies and various database and data warehousing technologies
  • Design for key issues such as complex ETL workflows, query optimization, and data caching across different backends for big data as well as streaming data
  • Address core data governance concerns such as access management, cataloging, versioning, logging, and auditing
  • Help establish best practices, document designs, and mentor junior team members
  • Support project management, define and estimate required work, track dependencies, report progress, highlight blockers
Requirements 
  • BA/BS preferred in a technical or engineering field
  • 3+ years experience
  • Good command of python or similar programming language as it relates to data operations (e.g. pandas, pyspark, pipelines)
  • Experience in data modeling and schema design and data lake construction
  • SQL experience and command of query design and optimization
  • Experience designing and automating ETL/ELT workflows using cloud services such as AWS data pipelines, Glue, EMR, and airflow or Azure DataFactory
  • Knowledge of different types of databases and when to use which (e.g. relational, document, key/value) as well as experience with particular implementations and cloud versions such as Snowflake, BigQuery, RDS, Redshift, Neptune, DynamoDB, Cosmos, Athena, etc.
  • Familiarity with data cataloging and warehousing technologies such as AWS Lake Formation, MSFT Synapse, and Alation
  • Familiarity with data engineering concepts such as batch vs stream processing and experience with technologies such as Kinesis and Kafka for stream processing
  • Experience with syntax like SQL, GraphQL, SPARQL, OQL, etc.
  • Familiarity with data types such as parque, avro, json, csv, xml, etc.
Preferences 
  • Awareness of cloud technologies such as serverless, autoscaling, etc.
  • Experience working in a regulated industry and meeting compliance requirements (e.g. SOC2, HIPAA, FDA, etc)
Join a team of passionate people committed to redefining the way business intelligence and AI is done.
 
For additional information, visit www.atscale.com