Principal Data Engineer

Houston, TX

Employment Type: Direct Category: Engineer Job Number: 18530 Work Model: #LI-Onsite Internal Reference: #LI-PF1

Job Description

Position Summary:
Our client, a hyper growth PE firm based in Houston is seeking a Principal Data Engineer who embodies strategic leadership and a practical, application-oriented mindset. This role is perfect for someone eager to make a substantial business impact utilizing modern cloud-based data processing technologies to ingest, curate and deliver clean datasets to business-critical processes and applications used across the business.

In this position, you will play the lead role implementing our data management strategy. You will focus on advancing our existing data platform, implementing the tools to increase security and governance, and manage the implementation and maintenance of our data processing pipelines. Your ability to translate complex technical subjects into clear business language will empower stakeholders to make informed decisions. You will ensure that our data platform can always support our immediate business objectives and that the future roadmap is closely aligned with our strategic and sustainable business goals. Your role goes beyond traditional data engineering; it's about harnessing the power of data to create tangible business outcomes. Collaborating with a larger offshore team, you will implement an industry leading data management solution that is designed to scale our business and deliver real-world results. Reporting to the Head of Engineering, you will establish yourself as a key figure in shaping  data-driven future, with a clear focus on practical applications and business impact.


Able to lead projects and work collaboratively with cross-functional teams, including data engineers, data analysts, subject matter experts and internal IT stakeholders.
Lead a team of onshore and offshore data engineers.

Define the to-be status of our data platform and lead implementation of the change process.

Responsible for ensuring that our data platform complies with our data governance policies.

Responsible for generating and reporting KPIs relating to service quality, ingestion, and consumption.

Responsible for delivering a “self-service” data access concept for key stakeholders, by implementing a secure but easy to use data consumption layer.

Manage the development and maintenance of new and existing data pipelines.

Conduct regular code reviews on bespoke pipeline code and notebook code.
Experience and Abilities:

10+ years' experience

Proven track record of implementing modern big data platform solutions, either Azure Data Bricks or Azure Synapse.

Terraform - Infrastructure as a Code

Prior experience with MDM and governance tools such as Unity Catalog or Microsoft Purview

Expert with Azure Data Lake, Avro, Parquet and Delta tables

Solid experience with Structured and Semi-Structured datasets

Solid understanding of ELT/ETL and the application using data pipelines

Ability to develop, implement and integrate notebooks within data pipelines.

Ability to setup and control data ingestion pipelines using Azure Data Factory or Data Bricks.

Prior experience ingesting public data sources such as Enverus, IHS, TGS

Extensive experience in common data engineering programming languages such as Python, R, SQL.

Extensive experience consuming and processing data from secure REST APIs

Comfortable integration data with data visualization applications using PowerBI, Spotfire, Plotly.

Familiar with Azure Dev Ops and CI/CD processes.

Strong communication skills and the ability to effectively propose and discuss solutions with data engineers as well as business partners at the appropriate level of technical detail.

BS, MS / PhD in Computer Science, or related disciplines, or a Petroleum Industry technical expert cross-trained through extensive work experience within the data engineering domain.

Energy domain expertise in Exploration & Production and/or Energy Transition & Decarbonization a plus.
Apply Online

Send an email reminder to:

Share This Job:

Related Jobs:

Login to save this search and get notified of similar positions.

About Houston, TX

Discover exciting job opportunities in the vibrant region around Houston, Texas! Known for its thriving economy, diverse industries, and endless growth prospects, Houston offers an enticing landscape for career-minded individuals. With world-class museums like the Museum of Fine Arts and a rich culinary scene showcasing Tex-Mex delights, BBQ joints, and food trucks serving up local flavors, Houston is a melting pot of culture and creativity. Home to the renowned Space Center Houston, the historic Astrodome, and professional sports teams like the Rockets and the Astros, this region exudes energy and excitement. Embrace the charm of this dynamic area while exploring job listings that could lead you to a fulfilling career right here in Houston!