Senior Data PySpark Engineer at Kforce Inc in Hartford, Connecticut

Job Description:

RESPONSIBILITIES:

Kforce has a client that is seeking a Senior Data PySpark Engineer in Hartford, CT.

Responsibilities:

• Leveraging cloud-based technologies for efficient data processing


• Improve and maintain data pipelines using PySpark for large-scale data processing and analysis


• Optimize data processing algorithms and systems for performance and efficiency


• Ensure data quality and integrity throughout the data processing lifecycle


• Handle CICD Operations through Jenkins & UDeploy or AWS Code build/Code Commit


• Collaborating with cross-functional teams to optimize data workflows

REQUIREMENTS:

• A minimum of 6-8 years of experience in Data Engineering


• Experience in AWS EMR, Snowflake experience is a plus


• Technical Leadership experience in creating conceptual, logical, and low-level designs, guiding and mentoring Junior team members & reporting risks/dependencies to leadership team


• Familiarity with AWS services such as S3, EMR, EC2, etc.


• Proficiency in handling large datasets, data pipelines, and distributed computing frameworks


• Proficiency in using PySpark data processing capabilities including optimization of spark jobs through partitioning, shuffling, broadcast variables, memory management


• Proficiency in using PySpark's data frame functionalities including Spark SQL


• Strong skills in PySpark for data processing, transformation, and analysis

The pay range is the lowest to highest compensation we reasonably in good faith believe we would pay at posting for this role. We may ultimately pay more or less than this range. Employee pay is based on factors like relevant education, qualifications, certifications, experience, skills, seniority, location, performance, union contract and business needs. This range may be modified in the future.

We offer comprehensive benefits including medical/dental/vision insurance, HSA, FSA, 401(k), and life, disability & ADD insurance to eligible employees. Salaried personnel receive paid time off. Hourly employees are not eligible for paid time off unless required by law. Hourly employees on a Service Contract Act project are eligible for paid sick leave.

Note: Pay is not considered compensation until it is earned, vested and determinable. The amount and availability of any compensation remains in Kforce's sole discretion unless
and until paid and may be modified in its discretion consistent with the law.

This job is not eligible for bonuses, incentives or commissions.

Kforce is an Equal Opportunity/Affirmative Action Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, pregnancy, sexual orientation, gender identity, national origin, age, protected veteran status, or disability status.

Apply Now