Required Skills:
- Databricks (Must-Have): 3+ years Workspace, Unity Catalog, Delta Live Tables (DLT), Workflows/Jobs, SQL Warehouses
- Delta Lake & Lakehouse: Lakehouse architecture design, Bronze/Silver/Gold patterns, ACID transactions, schema evolution
- Spark & PySpark (Must-Have): DataFrame API, Spark SQL, transformations, actions, performance tuning
- Streaming (Must-Have): Structured Streaming, Auto Loader for incremental ingestion
- Python (Must-Have): Production-grade scripting for data engineering and automation
- Azure Stack (Must-Have) ADF (orchestration), ADLS Gen2 (storage), Azure SQL (relational)
- Performance Tuning: Partitioning, Z-ordering, caching, shuffle optimization, job cluster sizing
Preferred Qualifications:
- Experience with Databricks Asset Bundles or CI/CD for Databricks (GitHub Actions, Azure DevOps).
- Familiarity with data modeling concepts (Star Schema, Data Vault 2.0).
- Exposure to Power BI / Tableau connecting via Databricks SQL.
- Knowledge of Databricks cost optimization (cluster policies, spot instances, photon engine).
- Databricks Certified Data Engineer Associate/Professional is a plus.
- Exposure to Agile/Scrum delivery methodologies.