Our client is seeking a Middleware Architect to join their team! This position is located in Irving, Texas.
Duties:
Design and build large scale data processing system to address growing AI/ML and Data needs
Build a product to process large amount data/events for AI/ML and Data consumption
Automate test coverage for data pipelines
Automate CI and deployment processes and best practices for the production data pipelines
Build AI/ML model based alert mechanism and anomaly detection system for the product
Desired Skills/Experience:
7+ years of overall experience in software development with 5+ years of relevant experience in designing, developing, deploying, and operating large data processing data pipelines at scale
3+ years experience with Apache Spark for Streaming and batch process
Good knowledge on Apache Kafka
Strong background in programming
Experience on building reusable data frameworks/modules
Experience on Airflow scheduler
Experience with Containers, Kubernetes, and scaling elastically
Strong background in algorithms and data structures
Strong analytical and problem solving skills
Strong bent toward completely automated code deployment/testing