Job Duties PySpark senior developer's responsibilities include the ability to design, build and unit test the applicationTranslate business and functional requirements into robust, scalable, operable solutions that work well within the overall data architecture at AvalaraDesign and develop data pipelines, data ingestion and ETL processes that are scalable, repeatable and secure for stakeholder needsParticipating in peer reviewsParticipating in Agile development activities including sprint planning, stand-ups, retrospectives and demosExcellent verbal and written communication skills to collaborate effectively with partner teams globally
QualificationBachelor of Engineering in Computer Science or equivalent5+ years experience designing and developing in pyspark/ PythonAtleast 2+ years of experience in AWS ETL ecosystem
Primary SkillsDeep working experience in developing data processing tasks using pySpark such as reading data from external sources, merge data, perform data enrichment and load in to target data destinationsWorking experience in AWS ecosystem services like S3/Athena AWS Athena/ Redshift/ Lambda/ S3/ Step functions, SQS, SNSExperience with data modelling and building ETL pipelinesExperience in data processing, aggregation and transformation with unit tests and design data processing pipeline, exception handling and performance optimisation techniqueStrong knowledge of working with different data file formats like Parquet, JSON, CSV, etcDefine and capture metadata and rules associated with ETL processesOrganise, cleanse, validate, and format data for storageWorking experience in SQL, NoSQL databases like – DynamoDB, Postgres, SQLExperience in GitLab deployment using Terraforms and operationalising the code is added advantageGood to have knowledge and skills in Devops/version control and containerisation
Added Advantage: AWS Certification
We're building cloud-based tax compliance solutions to handle every transaction in the world.
Imagine every transaction you make — every tank of gas, cup of coffee, or pair of sneakers, every movie ticket, meal kit, or streamed song, every sensor-to-sensor ping. Nearly every time you make a purchase, physical or digital, there's an accompanying unique and nuanced tax compliance calculation.
The logic behind calculating taxes — the rules, rates, and boundaries — is a global, layered, three-dimensional mess of complexity, with compliance dictated by governments and applied by every business, every day.
Avalara works with businesses of all sizes, all over the world — from corner stores to gigantic global retailers — to calculate tax accurately and automatically, at speeds measured in milliseconds.
That's a massive technical challenge, in terms of scale, reliability, and complexity, and we do it better than anyone. That's why we're growing fast.
Headquartered in Seattle, Avalara has offices across the U.S. and around the world, in Belgium, Brazil, Canada, India, and the U.K.
Avalara is an Equal Opportunity Employer. All qualified candidates will receive consideration for employment without regard to race, color, creed, religion, age, gender, national orientation, disability, sexual orientation, or any other factor protected by law.
Avalara is an Equal Opportunity Employer. All qualified candidates will receive consideration for employment without regard to race, color, creed, religion, age, gender, national orientation, disability, sexual orientation, US Veteran status, or any other factor protected by law.