Senior Data Engineer (Sitecore Search DataPlatform)
Description
About us:
Sitecore’s mission is to provide cutting-edge DXP solutions that enable the world’s greatest brands to craft truly unforgettable experiences for their customers. A highly decorated industry leader, Sitecore brings content, commerce, and data into one connected platform that delivers millions of digital experiences every day. Thousands of blue-chip companies, including American Express, Porsche, Starbucks, L’Oréal, and Volvo Cars, rely on Sitecore to provide more engaging, personalized customer experiences. Under the leadership of our new product-centric CEO, we are poised to continue to push the boundaries of marketing technology and shape the future of customer engagement. Learn more at Sitecore.com.
About the Role:
- Design, develop, and maintain a generic ingestion framework capable of processing various types of data (structured, semi-structured, unstructured) from customer sources.
- Implement and optimize ETL (Extract, Transform, Load) pipelines to ensure data integrity, quality, and reliability as it flows into the centralized datastore like Elasticsearch.
- Ensure the ingestion framework is scalable, secure, efficient and capable of handling large volumes of data in real-time or batch processes.
- Continuously monitor and enhance the data ingestion process to improve performance, reduce latency, and handle new data sources and formats.
- Implement large-scale near real-time streaming data processing pipelines.
- Build analytics tools that utilize the data pipeline to provide actionable insights into key business performance metrics.
What You Need to Succeed:
- Minimum 4 years of experience in a similar role, building and optimizing Big Data data pipelines, architectures and data sets
- Strong proficiency in Elasticsearch, its architecture and optimal querying of data.
- Experience working on Big Data processing systems like Hadoop, Spark, Spark Streaming, or Flink Streaming
- Hands-on experience in two or more of these integration technologies: Java/Python, React, Golang, SQL, NoSQL (Mongo), Restful API.
- Knowledge of at least one distributed datastores like MongoDb, DynamoDB, HBase
- Working knowledge of message queuing, stream processing, and highly scalable ‘big data’ data systems.
- Experience contributing to the architecture and design (architecture, design patterns, reliability and scaling) of new and current systems.
- Working experience on Big Data systems such as Hive, Oozie and Hadoop
Sitecore is proud to be an equal-opportunity workplace. We are committed to equal employment opportunity without unlawful regard to race, color, ancestry, religion, gender, national origin, sexual orientation, age, citizenship, marital status, disability, veteran status or any other local legally protected characteristic.