Data Engineer at ConnectedX Inc. in Plano, Texas

Posted in Other about 2 hours ago.

Type: full-time





Job Description:

Role: Data Engineer

Location: Plano, TX

What we are looking for

Highly experienced Full stack Engineer with expertise in cloud engineering, data engineering, and data warehousing, particularly within the AWS and google cloud ecosystem. Success in designing and implementing scalable cloud solutions, data pipelines, optimizing data architectures, and delivering high-quality analytics solutions. Proficient at collaborating with cross-functional teams to drive business insights and data-driven decision-making. Skilled in automating processes, and ensuring high availability and security in cloud environments.

implemented scalable cloud architectures for enterprise applications using AWS services for Data platforms

Develops and maintains CI/CD pipelines to automate deployment processes and ensure continuous integration.

Leads teams in designing and developing centralized data warehouses and data lakes. Preferably using redshift

Creates and optimizes ETL processes to ingest and transform large datasets for analytical purposes.

Utilizes Apache Airflow for scheduling and monitoring data workflows to ensure timely data delivery.

Implements data partitioning, compression strategies, and query optimization techniques.

Develops serverless applications using AWS Lambda and API Gateway to reduce operational costs.

Automates configuration management using tools like Ansible and Terraform.

Conducts security assessments and implemented IAM and VPC best practices to secure cloud environments.

Integrated monitoring and logging solutions using CloudWatch, Prometheus, and ELK Stack for performance and security monitoring.

Created automated reporting solutions and interactive dashboards using Tableau, Looker, and Power BI.

Conducts data quality checks and ensured data integrity across multiple data pipelines.

Ability to securely ingest data from public and other Toyota sources to better insights

Technical Skills:

Cloud Platforms: AWS (EC2, S3, RDS, Lambda, CloudFormation, VPC, IAM, Redshift)

Data Engineering: ETL, Data Warehousing, Data Modeling, SQL, NoSQL

Big Data Technologies: Hadoop, Spark, Athena, materialized views

DevOps Tools: Docker, Kubernetes, Jenkins, Terraform, Git, Big query, Firebase

Workflow Orchestration: Apache Airflow

Programming Languages: Python, SQL, Java, Go, Bash

Data Visualization: Tableau, Looker, Power BI, Quicksight

Monitoring & Logging: CloudWatch, Prometheus, Grafana, ELK Stack

Other Tools: GitLab CI/CDSecurity: Forerock, Keycloak, SSO solutions for data platforms
More jobs in Plano, Texas

Other
about 1 hour ago

Sam's Club
Other
about 2 hours ago

Sam's Club
Other
about 2 hours ago

Wipro
More jobs in Other

Other
2 minutes ago

Community Partners for Affordable Housing
Other
10 minutes ago

RIIM
Other
10 minutes ago

Beacon Hill