About the role
Lead the build-out of a unified lakehouse on Databricks for a healthcare analytics platform processing 4B+ clinical events monthly.
Key responsibilities
- Design medallion architecture (bronze/silver/gold) on Delta Lake with Unity Catalog governance.
- Author production-grade PySpark and SQL pipelines with DLT and Workflows.
- Lead a pod of 4 engineers across discovery, build, and release cycles.
- Partner with data science to productionize feature stores and ML pipelines.
- Own data quality, lineage, and HIPAA-aligned access controls.
Qualifications
- 8+ years of data engineering with 3+ years on Databricks or Spark at scale.
- Expert PySpark, SQL, and Delta Lake; strong Python.
- Experience with Unity Catalog, Workflows, and CI/CD on Databricks.
- Proven technical leadership of multi-engineer pods.
Technology stack
DatabricksPySparkDelta LakeAirflowSnowflake
Apply
Apply for Lead Data Engineer (Databricks)
Submit your details and resume. A senior recruiter responds within one business day.
Let's Build the Future Together.
Whether you're hiring a single engineer, scaling an enterprise technology team or transforming your digital infrastructure, BrightScale IT is ready to help.