Job Title - Azure DataBricks Engineer with Pyspark
Location - Raleigh NC | Richardson TX
Type:- Fulltime
Job Description:
Required Qualifications • Candidate must be located within commuting distance of Raleigh, NC or be willing to relocate to the area • Bachelor's degree or foreign equivalent required from an accredited institution. Will also consider three years of progressive experience in the specialty in lieu of every year of education. • At least 4 years of experience in Information Technology. • At least 3 years of hands-on experience in using Azure Data Factory (ADF) to orchestrate data ingestion from various sources into Azure Synapse and Azure Data Lake Storage (ADLS). • At least 2 years of hands-on experience with Azure, databricks and Hadoop distributed frameworks while handling large amount of big data using Spark and Hadoop Ecosystems. • At least 2 years of experience designing and implementing data pipelines using Azure Databricks for data cleaning, transformation, and loading into Azure Synapse Analytics • Hands on experience in end-to-end implementation of data warehouse and data marts • Experience in writing SQL queries to analysis Type-2 dimension data • Understanding of implementing Medallion Architecture using Azure Services • All applicants authorized to work in the United States are encouraged to apply .
Preferred Qualifications • Experience in software engineering and data engineering roles, with a focus on Azure and Databricks. • Strong understanding of databases and big data software technologies, specifically Azure and Databricks. • Exposure to Unity Catalogue • Experience with Agile methodologies, particularly Scrum. • Experience in coordinating with offshore data engineering teams • Experience with scripting languages like Python, Java, etc. • Strong problem-solving skills related to data, data structures, and algorithms