Job Description:
Roles and Responsibilities:Client is trying to build a streaming platform and this Requirementis for datbricks engineer.
TechStack (Must Have):AWS, Confluent Kafka Streaming, Pyspark, Delta Lake, Databricks(Medallion Architecture, DeltaLiveTables (DLT) with serverless, CDC(SCD Type 1 and Type2),Autoloader, Materialized Views, Unity Catalog, Workflows,DBT basic knowledge) , Autoloader - streaming experience,DevOps - CI/CD, Jenkins, Automation Guidance and service-based offering for the stream processing part,
TechStack (Good to Have):
- DataBricks Certified Dataengineer professional
- AWS Knowledge (Athena, EC2, S3)
- Trino SQL query engine
- PowerBi