Director, Data Engineering
San Francisco, CA · On-Site
Learn more about our Data & Analytics Recruitment solutions.
Axial Search recruits for AI transformation roles across North America, building long-term relationships with leaders across every industry vertical. Apply today to express your interest in this position and others like it. Visit our website to learn more about our process and to find free tools to support your job search — including our live job market dashboard with salary, skills, and hiring-trend data from thousands of AI transformation roles.
What the market looks like
We've tracked 500+ senior-level data engineering postings across the US in the last six months, with California, Texas, and New York accounting for the majority of open roles. The strongest candidates in this market bring deep experience building and scaling data platforms — data lakes, warehouses, pipelines, and real-time processing systems — often across financial services, technology, manufacturing, and healthcare sectors. Compensation for senior-level data engineering leaders typically ranges from $180,000 to $190,000 at the median, with top-of-market packages reaching $440,000. Leaders who stand out combine hands-on technical depth with the ability to architect for organizational scale, mentor growing teams, and partner across analytics, ML engineering, and product to unlock data as a strategic asset.
Job responsibilities
Own the strategic roadmap for data platform architecture and engineering, ensuring systems scale reliably as data volume and complexity grow across the organization
Lead a team of data engineers, setting technical direction, mentoring individual contributors, and building a culture of operational excellence and continuous improvement
Design and build robust data pipelines, warehouses, and real-time processing systems that serve analytics, ML, and product teams with reliable, well-documented access to data
Partner with data science, analytics, and ML engineering teams to understand data needs and ensure platform capabilities align with downstream use cases
Drive data governance, quality, and security standards across the organization, including schema design, metadata management, and access control frameworks
Evaluate and integrate new data technologies and cloud services, making build-vs-buy decisions that balance cost, performance, and team velocity
Establish metrics and monitoring for data platform health, latency, and reliability; own incident response and system optimization
Candidate requirements
8+ years of data engineering experience, including at least 3 years in a leadership or architectural role driving platform decisions and team outcomes
Hands-on expertise building and operating large-scale data systems — data warehouses, lakehouses, ETL/ELT pipelines, or real-time streaming platforms — in cloud environments (AWS, GCP, Azure)
Proven ability to hire, mentor, and grow engineering teams; comfortable setting technical strategy and holding teams accountable to quality and delivery standards
Strong foundation in SQL and at least one programming language (Python, Scala, Java, or Go); familiarity with modern data tools and frameworks (Spark, dbt, Airflow, Kafka, Snowflake, BigQuery, or equivalent)
Track record of defining and implementing data governance, quality, and security practices that scale with organizational growth
Excellent communication skills; ability to translate technical complexity for non-technical stakeholders and partner effectively across product, analytics, and ML teams
