DI – Data Engineer (CE50SF RM 3902)

Position: DI – Data Engineer (CE50SF RM 3902)

Primary skills : Data Engineer

Essential Responsibilities
Data Pipeline Development & Customization

• Design, develop, and maintain scalable data pipelines and workflows.
• Review custom scripts, business logic, and integrations for adherence to best practices.
• Utilize AWS and Azure development tools such as Glue, Lambda, Data Factory, and related APIs/SDKs.

Integration & Automation

• Design and implement integrations with SDKs alignment and Lambda compatibility
• Optimize workflows and automation using CI/CD pipelines and Infrastructure as Code
• Ensure seamless data flow between cloud platforms and ensure availability of data for the team.

Collaboration & Stakeholder Management

• Work closely with domain experts, developers, and product owners to translate business requirements into technical solutions.
• Provide consultation on AWS, Azure, and DevOps capabilities.
• Create and maintain dashboards communicating effectively to both technical and non-technical stakeholders.

Platform Maintenance & Upgrades

• Maintain cloud resource inventory (AWS and Azure subscriptions).
• Manage cloud platform upgrades, patching, and performance monitoring.
• Ensure minimal disruption and adherence to upgrade best practices.

Basic Qualifications

• 5+ years of total software engineering experience with strong focus on data engineering.
• Hands-on expertise in AWS and Azure cloud services for data processing and storage.
• Strong experience in DevOps practices, CI/CD pipelines, and Infrastructure as Code.
• Proficiency in Python, SQL, and scripting languages.
• Understanding of cloud infrastructure, security, and compliance frameworks.

*******************************************************************************************************************************************