Role: Data Engineer

Location : 100% Remote

Job Type : Contract

Visa : only USA Citizen

Primary Skill:

AWS Services, data lakshould, Lakformation, SQL Server

Secondary Skill:

IAM S, EKS

Job Description:

Automate creation, transformation, and integration of raw and enriched data models in the data Lakehouse. Implement automated ETL pipelines for data ingestion, transformation, and consumption Design and implementation for integrating Lakeformation across AWS Data Platform Accounts.

Design and implementation for integrating Lakeformation with Keycloak.

Design and implementation for an integrated IAM solution including EKS Service Accounts.

Design and implementation of HashiCorp Vault with integration to AWS Services such as Secrets Manager, Certificate Manager and Keycloak.

Development of integration of HashiCorp Vault with AWS Data Services such as RDS Aurora PostgreSQL, RDS MSSQL and other RDS solutions as required.

Provision MSSQL server on AWS using Terraform

Deploy read replica for MSSQL server using Terraform

Implement monitoring & logging for MSSQL server using Terraform

Transition existing pipeline to MSSQL server

Collaborate with the business application owner on the existing data architecture, including data ingestion, data pipelines, business logic, data consumption patterns, and analytics requirements

Design and document the target data architecture, pipelines, processing and analytics architecture

Identify opportunities for optimization and consolidation

Collaboration with data team on decomposition of business logic and data transformation pattern.