Senior Data Engineer
Description
Company Overview
Lean Solutions Group is committed to enhancing business efficiency through innovative technology solutions. Our mission is to drive transformation by leveraging cutting-edge AI and machine learning techniques. We uphold a culture of continuous learning, collaboration, and customer-centricity, ensuring exceptional outcomes for our clients. Recognized for our expertise in deploying scalable solutions that seamlessly integrate with enterprise systems, we have established a strong foothold in the tech industry. With a global presence and a team of dedicated professionals, Lean Solutions Group is at the forefront of innovation, constantly pushing boundaries to meet the dynamic needs of businesses worldwide.
What You Will Be Doing
- Architect and implement autonomous GPT-based systems for real-time workflow automation.
- Fine-tune and optimize large language models for specific automation tasks.
- Automate end-to-end processes managing unstructured data and decision logic.
- Develop and maintain data pipelines using AWS services like Glue and Lambda.
- Ensure data integrity and efficient processing using tools like Pandas and PySpark.
- Design scalable solutions for seamless integration with enterprise cloud systems such as EC2 and RDS.
- Implement monitoring systems for deployed models and incorporate feedback mechanisms.
- Collaborate with cross-functional teams to gather requirements and deliver solutions aligned with business objectives.
- Stay informed on industry trends in AI and machine learning, integrating innovations into current systems.
Required Skills & Experience
To excel in this role, you should possess:
To excel in this role, you should possess:
- Bachelor's or Master's degree in Computer Science, Machine Learning, Data Engineering, or a related field.
- 5+ years of experience in data engineering and machine learning, particularly with NLP and process automation.
- Advanced proficiency in Python, including experience with data analysis using pandas and machine learning frameworks like PyTorch and TensorFlow.
- Intermediate experience with AWS services, such as RDS, EC2, Glue, and Lambda, for managing data solutions and scaling models.
- Deep understanding of LLMs with experience in fine-tuning and applying NLP techniques.
- Experience with API development and integration.
- Strong knowledge of MLOps practices, including model deployment, versioning, and monitoring.
- Familiarity with data engineering tools and frameworks such as Pandas and PySpark for efficient data processing.
- Excellent problem-solving skills and ability to work with cross-functional teams.
- Intermediate knowledge of Docker and familiarity with microservices architecture and containerization technologies.
- Understanding of prompt engineering and reinforcement learning from human feedback (RLHF) to enhance GPT-based systems.
- Experience with reinforcement learning from human feedback (RLHF) techniques
- Familiarity with microservices architecture
- Proficiency in utilizing pandas and PySpark for data analysis
- Experience with process automation tools and frameworks
- Knowledge of prompt engineering to enhance GPT-based systems
- Understanding of the ethical considerations and safety measures in autonomous systems
- Intermediate experience with Docker for containerization
- Advanced problem-solving skills with innovative solution approaches
- Effective Communication: Ability to clearly articulate complex technical concepts to non-technical stakeholders, facilitating cross-functional collaboration.
- Problem-Solving: Strong analytical and innovative skills, essential for navigating and resolving complex challenges in data engineering and process automation.
- Collaboration: Demonstrated experience working with diverse teams to gather requirements and deliver data-driven solutions aligned with business objectives.
- Adaptability: Proactively stay informed about industry trends in AI and machine learning, and integrate innovative solutions into current systems to maintain a competitive edge.
- Leadership: Lead projects with a focus on optimizing performance and efficiency, fostering a culture of continuous improvement and learning.
Why You Will Love Working with Us
- Join a powerful tech workforce and help us change the world through technology.
- Professional development opportunities with international customers
- Collaborative work environment Career path and mentorship programs that will lead to new levels.
Join Lean Tech and contribute to shaping the data landscape within a dynamic and growing organization. Your skills will be honed, and your contributions will play a vital role in our continued success. Lean Tech is an equal-opportunity employer. We celebrate diversity and are committed to creating an inclusive environment for all employees.