We are focused on leveraging technology to drive data insights and innovation. We are seeking a skilled Data Engineer with experience in Google Cloud Platform (GCP), SQL, and ETL processes to join our growing data team. This role offers the opportunity to work on cutting-edge cloud-based solutions, build scalable data pipelines, and contribute to impactful data-driven projects.
As a Data Engineer, you will be responsible for designing, building, and maintaining efficient data pipelines and ETL processes on Google Cloud Platform. You will work closely with data analysts, data scientists, and software engineers to ensure the availability, quality, and integrity of data for reporting and analytics. Your expertise in SQL and GCP services will play a key role in transforming raw data into valuable insights that support strategic decision-making.
Key Responsibilities:
Data Pipeline Development: Design, develop, and maintain scalable ETL pipelines to ingest, process, and store large volumes of structured and unstructured data from multiple sources.
Google Cloud Platform (GCP) Expertise: Utilize GCP services such as BigQuery, Cloud Storage, Dataflow, and Pub/Sub to build, optimize, and manage cloud-based data infrastructure.
SQL Querying: Write and optimize complex SQL queries for data extraction, manipulation, and reporting to ensure efficient access to data across different systems.
Data Integration: Collaborate with cross-functional teams to integrate data from diverse sources including APIs, databases, and external platforms into a unified data warehouse.
Data Warehousing: Implement and manage cloud-based data warehousing solutions using Google BigQuery or similar platforms to store and organize large datasets for easy accessibility.
Data Quality and Governance: Ensure data accuracy, consistency, and security through the implementation of data quality checks, monitoring, and governance practices.
Performance Tuning: Optimize database performance by analyzing and tuning SQL queries, ETL processes, and data pipeline workflows for efficiency and scalability.
Automation: Develop automated data workflows and monitoring systems to ensure timely data availability and minimize manual intervention.
Collaboration: Work closely with data scientists, analysts, and business stakeholders to understand data requirements, provide insights, and enable data-driven decision-making.
Documentation: Create and maintain comprehensive documentation for data pipelines, ETL processes, and database structures to ensure team alignment and knowledge sharing.
Qualifications:
Education: Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field. Master's degree is a plus.
Experience:
3+ years of experience as a Data Engineer or similar role, working with data pipelines, ETL processes, and cloud technologies.
Hands-on experience with Google Cloud Platform (GCP) services such as BigQuery, Dataflow, Cloud Storage, and Pub/Sub.
Proficiency in writing complex SQL queries and optimizing them for large datasets.
Experience with ETL tools and processes to transform and integrate data from multiple sources.
Strong understanding of data warehousing concepts, database design, and data modeling.
Technical Skills:
Advanced knowledge of SQL for querying, analyzing, and manipulating data.
Experience with Python, Java, or other programming languages for data processing tasks is a plus.
Familiarity with GCP's IAM policies, data security, and cost management.
Experience with Airflow or other orchestration tools to automate workflows.
Strong understanding of ETL frameworks, data lakes, and data warehousing solutions.
Knowledge of version control systems like Git and CI/CD pipelines.
Soft Skills:
Strong analytical and problem-solving skills with the ability to work with complex data sets.
Excellent verbal and written communication skills, with the ability to collaborate effectively with both technical and non-technical stakeholders.
Detail-oriented with a focus on data accuracy and quality.
Ability to work independently and in a collaborative team environment.
Preferred Qualifications:
Experience with Apache Spark, Dataproc, or other distributed data processing frameworks.
Familiarity with NoSQL databases such as Firestore or MongoDB.
Certifications in Google Cloud Platform (e.g., Google Professional Data Engineer).
Experience with machine learning pipelines and data science workflows.