Cloud Proficiency : Google Cloud Platform (GCP) services, such as BigQuery, CloudRun, CloudStorage, Pub/Sub, Workflows, IAM, Etc . Understanding of cloud-native solutions and architecture is critical.
Data Engineering Transformation Tools: Familiarity with data integration tools Cloud Composer (based on Apache Airflow) , Dataflow , CloudRun.
Programming Languages/Infrastructure as Code/ Code Management : Strong knowledge of languages commonly used in data engineering, such as Python, SQL ,Terraform, CI/CD , Git flows, Attunity , AutoSys
Data Modeling and ETL Processes: Ability to design robust data models and develop efficient ETL (Extract, Transform, Load) processes, ensuring data quality and consistency.
Monitoring and Optimization: Very imp - Skills in monitoring data pipelines and GCP resources using Stackdriver (Google Cloud Operations Suite), and optimizing performance and cost.
Problem-Solving/Debugging Skills: Very imp - Strong troubleshooting and problem-solving skills to quickly resolve issues that arise during production. Which includes reviewing log , working with networking , security , IAM , DBAs.
Communication Skills: Effective communication skills for collaborating with global teams, including clear documentation practices and the ability to convey technical details to non-technical stakeholders.
Automation: Experience in automating routine data processing tasks using scripts and GCP tools to improve efficiency and reliability.