Job Description
Position
Senior Associate Projects (Big Data Engineer)
Experience
6 ? 10 Years
Location
Chennai (Hybrid)
Shift
Day Shift
Job Summary
We are seeking a highly skilled Big Data Engineer with expertise in Hadoop, Spark, Python, PL/SQL, and ETL Tools to design, develop, and optimize large-scale data processing solutions. The ideal candidate will be responsible for building scalable data pipelines, ensuring data quality, and enabling data-driven decision-making across the organization.
The role requires hands-on experience in big data technologies, data integration, data transformation, and performance optimization within enterprise environments.
Mandatory Skills
- Hadoop
- Apache Spark
- Python
- PL/SQL
- ETL Tools
- Data Gap Analysis
- Data Integration
- SQL
Key Responsibilities
- Design, develop, and maintain scalable data pipelines using Hadoop and Spark.
- Build and optimize ETL workflows to extract, transform, and load data from multiple sources.
- Perform data gap analysis and implement solutions to improve data quality and consistency.
- Develop complex PL/SQL procedures, functions, packages, and performance-tuned queries.
- Use Python to automate data processing, workflow orchestration, and reporting activities.
- Monitor, troubleshoot, and optimize big data applications and ETL jobs.
- Collaborate with business stakeholders and technical teams to gather requirements and deliver data solutions.
- Ensure data integrity, security, and compliance with organizational standards.
- Participate in code reviews and implement best practices for scalable data engineering solutions.
- Improve system performance through tuning, optimization, and process enhancements.
- Document technical designs, data flows, and operational procedures.
Required Qualifications
- 6?10 years of experience in Data Engineering or Big Data Development.
- Strong hands-on experience with Hadoop ecosystem components.
- Expertise in Apache Spark for large-scale data processing.
- Experience with ETL tools and data integration projects.
- Strong proficiency in Python programming.
- Advanced knowledge of PL/SQL and database optimization techniques.
- Experience working with structured and unstructured data.
- Strong analytical, troubleshooting, and problem-solving skills.
- Excellent communication and stakeholder management skills.
Preferred Certifications
- Certified Hadoop Developer
- Apache Spark Certification
- Python Programming Certification
Why Join Us?
- Work on enterprise-scale Big Data platforms.
- Exposure to cutting-edge data engineering technologies.
- Opportunity to collaborate with cross-functional teams on high-impact projects.
- Hybrid work environment with excellent growth opportunities.