We are seeking a skilled Data Engineer with strong Python expertise to design, build, and optimize scalable data pipelines and architectures. The ideal candidate will work closely with data scientists, analysts, and business teams to ensure reliable data availability and quality.
Key Responsibilities
- Design, develop, and maintain robust data pipelines using Python
- Build scalable ETL/ELT workflows
- Work with large datasets using Pandas, PySpark, or Dask
- Integrate data from multiple sources
- Optimize data storage and retrieval
- Ensure data quality and validation standards
- Collaborate with cross-functional teams
- Deploy and monitor pipelines using Airflow
- Work on cloud platforms like AWS, Azure, or GCP
Required Skills & Qualifications
- Strong proficiency in Python
- Experience with SQL and relational databases
- Hands-on ETL experience
- Familiarity with Spark or Hadoop
- Knowledge of data warehousing
- Experience with REST APIs
- Understanding of data structures and algorithms
Preferred Skills
- Cloud platform experience
- Docker/Kubernetes
- Kafka or stream processing
- CI/CD for data workflows
- Basic ML pipeline understanding