Role: Data bricks Developer
Industry Type: IT Services & Consulting
Department: IT & Information Security
Employment Type: Full Time, Permanent
Role Category: IT & Information Security - Other
UG: Any Graduate
Job Title: Data Bricks Developer
Location: Hyderabad
Work mode: Hybrid
Shift Time: 1:00 PM 10:00 PM
Key Responsibilities
Data Pipeline Development
- Design and develop scalable data pipelines using Azure Databricks, Python, and PySpark.
- Implement ETL/ELT workflows for structured, semi-structured, and unstructured data.
- Develop and maintain data processing workflows using Delta Lake architecture.
- Ensure efficient data ingestion, transformation, and loading processes.
Data Management & Optimisation
- Optimise performance of data pipelines and processing workloads.
- Ensure data quality, reliability, and consistency across data platforms.
- Write and optimise complex SQL queries for data processing and validation.
- Implement data governance and access controls.
Cloud & Data Platform Integration
- Work with Azure Data Lake for storing and managing large datasets.
- Manage data cataloging and governance using Unity Catalog.
- Integrate data workflows with CI/CD pipelines for automated deployments.
- Support scalable and secure cloud-based data architectures.
Collaboration & Migration Projects
- Work closely with cross-functional teams to deliver data-driven solutions.
- Participate in data migration and modernisation initiatives.
- Support troubleshooting and resolution of data pipeline issues.
- Document data architecture, pipelines, and processes
Required Skills & Experience
- Must have 7 years of IT experience and at least 5+ years of hands-on experience in Databricks.
- Design and develop scalable data pipelines using Azure Databricks, SCALA, Python, PySpark, and Delta Lake.
- Have a good understanding of RDBMS and proficiency in writing complex SQL (1st Preference Oracle / PLSQL).
- Implement ETL/ELT workflows for structured, semi-structured, and unstructured data.
- Optimise performance of data processing and ensure data quality standards.
- Work with Azure Data Lake, Unity Catalogue, and integrate with CI/CD pipelines.
- Strong knowledge of Databricks technologies, specifically Delta Lake and Apache Spark, with a focus on implementing data engineering best practices.
- Experience with orchestration tools like Airflow or Azure Data Factory is a plus.
- Have working experience on a migration project.
Relevant Experience Preferred
- Hands-on experience developing data pipelines using Azure Databricks and PySpark.
- Experience implementing ETL/ELT solutions for enterprise data platforms.
- Experience optimizing data processing performance and data workflows.
- Experience working with Azure Data Lake and cloud-based data architectures.
- Experience working on data migration or modernization projects.
- Experience integrating data pipelines with CI/CD workflows.
Interested candidates can share their resumes to [email protected]