Data Engineer
Description
Data Engineer
Reports To: Director, Product Development & Operations
BAI is the financial services industry’s leading provider of powerful tools, relevant content, and meaningful connections. Our mission is to give financial services leaders the confidence to make smart business decisions because a strong financial services industry helps consumers, businesses and communities thrive.
We are looking for a passionate and experienced Data Engineer to help us transform our class-leading regulatory compliance and online training platforms using cutting-edge cloud-native concepts on the Microsoft Azure Platform.
Responsibilities:
- Build systems, algorithms, and analytical tools to transform data into actionable insights that drive business value.
- Design, deploy, maintain, and ensure the consistency of the organization's databases and database architectures.
- Design, build, manage, and optimize data pipelines to support ETL processes and data integration tasks in a multi-tenancy cloud data platform using Azure and SQL technologies.
- Ensure all solutions meet high levels of performance, security, scalability, maintainability, availability, sustainability, appropriate reusability, and reliability.
- Optimize data processing through database tuning, query optimization, and ensuring scalable infrastructures.
- Develop and deploy robust data and reporting architectures (e.g., data lake, data warehouse) to handle large-scale datasets and ensure data quality and integrity.
- Identify, design, and implement internal process improvements including re-designing infrastructure for greater scalability, optimizing data delivery, and automating manual processes.
- Collaborate with software engineers and product managers to understand detailed requirements for creating and enhancing new and legacy reporting and analytics solutions while adhering to architectural requirements of maintainability and scalability.
- Create new data validation methods and data analysis tools.
- Ensure compliance with data governance and security policies.
- Use and advise on specific technologies and best practices for utilizing cloud resources to efficiently ingest and process data and solve complex data problems.
- Administer databases, including modeling, design, and troubleshooting.
- Lead efforts to assess new technologies, products, and tools that can help improve business capabilities, customer value, and/or developer efficiency, and recommend specific solutions.
- Establish, document, and ensure compliance with data management best practices.
- Maintain comprehensive documentation of data models, pipelines, and ETL processes.
Requirements:
- Bachelor’s degree in computer science or related field strongly preferred
- Minimum of 5 years of experience as a data engineer
- Certifications in Azure Data Engineering, Spark, or other relevant technologies preferred
- Proven track record of developing ETL processes, data pipeline architecture, data models, and data analytics/reporting solutions
- Experience designing, implementing, and optimizing data architecture solutions in Azure
- Experience with SQL and ability to understand and author complex SQL queries, stored procedures, and functions
- Proven track record of leveraging data to deliver business value and present data-driven insights to business audiences
- Drive innovation in collaboration with business and technical stakeholders in alignment with the organization’s needs, overall applications portfolio, IT and business priorities and success measures to design and implement data solutions
- Communicate with all levels and functional areas of the BAI organization.
- Write and maintain documentation as needed to support knowledge sharing of existing and future technologies.
- Ability to understand business requirements and convert into technical specifications
- Ability to effectively communicate technical concepts to both technical and non-technical stakeholders
- Self-starter, comfortable with ambiguity, ability to initiate and drive projects with minimal oversight and guidance
- Excellent collaboration, communication, and leadership skills
- Highly motivated, self-directed, and possess solid execution and follow-up skills; detail oriented but understands the broader impacts of projects and tasks
BAI offers an outstanding benefit package including comprehensive insurance coverage, a 401(k) plan with company match, generous vacation/personal time programs, BAI Workflex which includes a hybrid work model and bonus potential with the ability to work in a collaborative, team-oriented environment.