Job Description

Salary: $71,000 - 111,000 per year Requirements:

• We are looking for a candidate with 8+ years of hands-on experience in data engineering, big data development, or a closely related field.
• We require extensive practical experience with Azure Databricks and PySpark.
• We need a strong programming background and deep knowledge of distributed data processing beyond traditional ETL tools.
• We expect proven expertise in Spark performance tuning and optimization.
• We expect proven expertise in cost management within Databricks.
• We expect proven expertise in Structured Streaming for real-time data processing.
• We expect proven expertise in Lakehouse Declarative Pipelines (LDP) and Delta Live Tables (DLT).
• We prefer familiarity with the Azure data ecosystem, including ADLS, Azure Data Factory, and Synapse.
• We value excellent communication skills and the ability to collaborate effectively with cross-functional teams.
• We require willingness and ability to work onsite in Houston.
• Nice to have: experience in energy, utilities, or heavy industry.
• Nice to have: knowledge of data governance, security, and monitoring.

• We will lead the design and development of scalable, high-performance data pipelines, streaming tables, and Delta Live Tables using Azure Databricks and PySpark.
• We will drive Spark performance tuning and apply cost-optimization strategies within the Databricks environment.
• We will build and manage real-time and batch workflows using Structured Streaming.
• We will leverage Delta Live Tables and Lakehouse Declarative Pipelines to create scalable, reliable, and maintainable data pipelines.
• We will collaborate with data architects, analysts, and business stakeholders to deliver robust data solutions.
• We will provide technical leadership through mentorship, code reviews, and architectural guidance.
• We will help ensure system reliability and performance through proactive monitoring and engineering best practices.

• Azure
• Big Data
• Databricks
• ETL
• PySpark
• Security
• Spark
• Cloud

More:

We are hiring an onsite Databricks Lead in Houston for a full-time role. This is a hands-on senior position for a programming-first engineer who can build high-performance, cost-optimized data solutions and provide technical leadership. We work closely across teams to deliver robust, scalable data platforms with a strong focus on Spark internals, declarative pipeline development, and real-time data processing.

last updated 38 week of 2026

Job Tags

Similar Jobs