Job Title: Senior Data Engineer

Duration : Long Term

Location: Remote

NOTE: Need only Mid level experience,   Must Have skills : Talend + Snowflake developer with prior Informatica or Ab initio knowledge.

Senior Data Engineer (SQL Server, SSIS & Databricks) – Offshore Coordination

We are looking for a Senior Data Engineer with expertise in SQL Server, SSIS, and Databricks to lead our data migration initiatives and ensure smooth offshore coordination. This role is ideal for a highly skilled data professional with deep technical experience in ETL development, data transformation, and cloud-based data architectures who can also manage offshore teams and collaborate with business stakeholders.

As a Senior Data Engineer, you will play a key role in migrating our data pipelines from SQL Server to Databricks, optimizing performance, and implementing best practices for scalable, cost-effective data solutions. You will work closely with onshore and offshore teams, ensuring seamless execution of deliverables while maintaining high-quality standards.

Responsibilities:

Lead the migration of SQL Server views and SSIS packages to Databricks, designing efficient ETL pipelines using PySpark, Delta Lake, and SQL.

Architect and implement scalable, high-performance data solutions within Databricks, ensuring optimal cost monitoring and governance.

Oversee data ingestion from various sources, including file uploads, Oracle, and external systems, ensuring seamless data integration and transformation.

Act as the primary liaison for offshore teams, managing workflows, conducting code reviews, and ensuring project milestones are met.

Collaborate with business stakeholders to understand data requirements, validate outputs, and provide training on new data platforms.

Implement logging, monitoring, and alerting mechanisms to ensure the reliability of data pipelines.

Continuously optimize query performance, storage strategies, and data pipeline efficiency to improve processing speed and cost-effectiveness.

Maintain clear documentation of data lineage, transformation logic, and architectural decisions to support ongoing data governance efforts.

Required Qualifications:

7+ years of experience in SQL Server development with expertise in SSIS, stored procedures, performance tuning, and data transformations.

3+ years of hands-on experience with Databricks, including PySpark, Delta Lake, and SQL-based ETL development.

Strong understanding of data modeling, performance optimization, and cloud-based data solutions.

Proven experience in offshore team coordination, managing deliverables, and ensuring quality standards.

Ability to work with data from multiple sources, including structured and semi-structured formats (file uploads, Oracle, APIs, etc.).

Excellent communication and stakeholder management skills to bridge the gap between business and technical teams.

Familiarity with cloud platforms such as Azure, AWS, or GCP, with a focus on data engineering best practices.

Experience with DevOps for Databricks, including CI/CD, automation, and cost governance, is a plus.

Preferred Skills:

Certification in Azure Databricks or Azure Data Engineering.

Experience with Airflow, ADF (Azure Data Factory), or other orchestration tools.

Knowledge of big data processing, distributed computing, and enterprise data governance