Lead Big Data Engineer (GCP Cloud)
at InfoVision Inc.
in
Irving,
Texas
Posted in Other 2 days ago.
Type: full-time
Job Description:
Job title: Lead Big Data Engineer (GCP Cloud)
Location: Irving, TX
Duration: Long-term
Skills Needed:
10+ years of experience in designing and building data pipelines in large-scale distributed systems.
Lead the design, development, and maintenance of scalable batch and real-time data processing pipelines.
Proficiency with Google Cloud Platform (GCP) and tools such as BigQuery, Dataflow, Pub/Sub, GCS.
Strong experience with the Hadoop Big Data Ecosystem: HDFS, Hive, Pig, HBase, YARN, and others.
Proficiency in Python and Scala for data engineering and scripting.
In-depth knowledge of SQL for data manipulation, transformation, and analysis.
Experience building both batch and real-time streaming data pipelines
Solid understanding of Hadoop fundamental concepts
Strong familiarity with CI/CD pipelines
Excellent problem-solving skills with an ability to troubleshoot complex data systems and performance bottlenecks.
Proven ability to lead and mentor teams of engineers, driving the development of cutting-edge data solutions.