Role: Data Science & Analytics - Other
Industry Type: IT Services & Consulting
Department: Data Science & Analytics
Employment Type: Full Time, Permanent
Role Category: Data Science & Analytics - Other
UG: Any Graduate
Execute priority workload migrations across all 3 tracks (L&S, Refactor, New-Build) building production-grade pipelines on Databricks and Snowflake, running dual-run validations, and remediating technical debt across schemas, naming, and lineage.
Key Responsibilities
Execute automated Lift & Shift migrations schema mapping, Iceberg table creation, pipeline generation for Bronze/Silver tables. Develop and refactor ETL/ELT pipelines decompose monolithic PSQL/Glue jobs into modular PySpark/Snowpark patterns. Review, validate, and refine AI-assisted code conversions (PySpark/Snowpark); label AI-generated lines in code comments per EchoStar Addendum. Configure and execute dual-run validation scripts row-count, hash-based integrity, business rule comparisons. Rationalize schemas, naming conventions, and lineage for AI Factory consumption patterns and Unity Catalog / Horizon Catalog governance. Develop and deploy Airflow DAGs replacing Control-M orchestration. Produce before/after metrics (complexity, runtime, cost) for Technical Debt Remediation Log. Create operations runbooks, SOPs, and procedural guides for each migrated workload. Support EchoStar FTE shadowing and reverse shadowing during KT phases.
Must-Have: Skills:
7-10 years in data engineering with cloud platforms. Strong proficiency in SQL, Python, PySpark, Apache Spark. Hands-on experience with Databricks (Delta, DLT, Autoloader, Unity Catalog) and/ OR Snowflake (Snowpark, Snowpipe, Streams, Tasks).
Experience: with AWS services รน S3, Redshift, Athena, Glue, EMR, Lambda.
Hands-on with Apache Airflow DAG development.
Experience: with GitLab for version control and CI/CD.
Strong SQL performance tuning and optimization skills.
Experience: with data quality validation and reconciliation.
Nice-to-Have:
Experience: migrating from Redshift/Athena to Databricks or Snowflake.
Familiarity with Control-M and Control-M Airflow migration.
Experience: with Qlik Replication / AWS DMS.
Shell scripting (Linux/Unix). AWS or Databricks certifications.