We are seeking a highly skilled and experienced Senior Data Engineer to join our dynamic team. This role requires a strong background in data engineering, cloud platforms, and healthcare data, along with expertise in modern software engineering and data orchestration tools. The ideal candidate will contribute to building and optimizing data pipelines, managing cloud-based infrastructures, and collaborating with cross-functional teams to deliver high-quality solutions.
Responsibilities:
Design, develop, and maintain scalable data pipelines using SPARK, Python, and SQL.
Implement robust ETL processes and ensure seamless data integration.
Leverage cloud platforms such as Azure, AWS, GCP, and tools like Databricks, Snowflake, and Microsoft Fabric to manage and optimize data infrastructure.
Work with healthcare data, ensuring compliance with industry standards and providing insights to drive business decisions.
Use data orchestration tools like ADF, Airflow, and DBT to automate workflows and ensure data availability.
Implement CI/CD pipelines using tools such as Jenkins, GitHub, and Maven for streamlined deployments and testing.
Required Qualifications:
4+ years of experience in software engineering with expertise in SPARK, Python, and SQL.
4+ years of hands-on experience in data engineering, including ETL and pipeline building.
2+ years of experience working with cloud platforms like Azure, AWS, or GCP, including tools like Databricks, Snowflake, or Microsoft Fabric.
1+ years of experience handling healthcare data and understanding its unique requirements.
Experience with data orchestration tools such as ADF, Airflow, or DBT.
Proficiency in CI/CD tools such as Jenkins, GitHub, and Maven.
Preferred Qualifications:
Knowledge of healthcare concepts such as claims, eligibility, benefits, pricing, underwriting, stop-loss reinsurance, and reserves.
Experience building business intelligence solutions using tools like Power BI or Tableau.
Proven experience leading a team and managing engineering projects.
Experience in designing and building engineering processes in collaboration with actuaries and/or data scientists.