As part of our agile development team, we are seeking a self-motivated Lead Data Engineer who excels at creative problem solving, possesses effective communication skills, and thrives on collaboration with team members. The Lead Data Engineer will be responsible for gathering and translating business and service requirements into queries, models and tools to allow for insightful data analysis and forecasting. You'll collaborate with various users to develop and interpret reports, create complex data queries, reconcile data from sources, develop ad-hoc queries to augment/pilot new reports and perform data cleansing and auditing to identify idiosyncrasies and ensure data integrity.
You will be entrusted with the critical task of ensuring data quality, overseeing the data pipeline development, integrate new technologies, and collaborate with stakeholders to align data solutions with business needs, driving data initiatives and enabling data-driven decision-making. Additionally, you will help lead a team consisting of Data Analysts, Data Developers and Data Engineers. You will also optimize system performance and ensure compliance with data governance standards. This role requires experience in working with SQL databases, writing and executing SQL queries and using the results in the creation of technical/business reports, understanding of reporting tools/services such as PowerBI, Tableau, StarQuery, etc.
Role Responsibilities:
Work with business units such as users, managers, supervisors and team lead to understand the challenges and map out and implement a timely solution
Be the point of contact for report/analytic needs
Monitor data integrity to drive the creation and maintenance of valid, standardized data for accounts, contacts, and transactions
Work across multiple departments to gather data and business requirements to meet reporting and development needs
Overlook all data needs and keep up on all development that requires resources
Work with the Analyst and DBA to ensure data is concise and the reporting tools are working as required
Help lead a team of DBA's, Data Developers, and Data Analysts
Be the go-to person for data related questions
Work with vendors to supply essential information as requested
Provide leadership, guidance, and supervision to direct reports, support staff, and other IT department staff regarding projects and/or technical tasks and assignments
Design, develop and implement database systems in support of business needs
Assist with the interviewing, hiring, and training of Data team members
Develop and implement data architecture and strategy
Design, build, and maintain data pipelines
Integrate new data technologies and tools
Ensure compliance with data governance standards
Foster professional growth within the team
Oversee data engineering projects from start to finish helping the team coordinate resources, processes, and give direction through ADO (Azure DevOps)
Experience
PL/SQL experience
Report writing experience with SQL Developer or other reporting tools
5+ Years experience as a Data Analyst, DBA, Data Developer or like field
3+ years' experience in a Data leadership role
Excellent written and verbal communication skills
Experience with Data Analytics and Predictive Modeling solutions (such as Tableau).
Business case development
Report writing experience with SQL Developer or other reporting tools
Experience MS SQL Server
Experience establishing a data platform, including a data lake and data warehouse, preferably in a cloud environment.
Advanced Excel skills required
Experience leading technical teams
Strong background in Math
Basic HTML/Web Programming experience
Experience incorporating external datasets into the organization, including monitoring of how those datasets are used
Strong knowledge of data architecture, data modeling, and ETL processes
Proficiency in programming languages like Python, Java, or Scala
Experience with data storage technologies (SQL, NoSQL databases)
Familiarity with big data tools and platforms (Hadoop, Spark, Kafka)
Strong problem-solving and analytical skills
Preferred Skills
Experience with cloud platforms (AWS, Azure, Google Cloud) - we are a MS shop, so Azure is preferred
Ability to work closely with product owners, web developers, UX designers and DBA's
Knowledge of data warehousing solutions (Redshift, Snowflake, BigQuery)
Familiarity with machine learning and AI concepts
Experience with containerization and orchestration (Docker, Kubernetes)
Strong understanding of DevOps practices and CI/CD pipelines
Advanced SQL and database optimization skills
Knowledge of scripting languages (Shell, Perl)
Ability to manage multiple projects and priorities
Strong business acumen and ability to align data initiatives with organizational goals