Data Engineer (AWS QuickSight, Glue, PySpark) (Noida) (CE46SF RM 3386)

Position: Data Engineer (AWS QuickSight, Glue, PySpark) (Noida) (CE46SF RM 3386)

Education Required: Bachelor’s / Masters / PhD:

• Bachelor’s or master’s in computer science, Statistics, Mathematics, Data Science, Engineering
• AWS certification (e.g., AWS Certified Data Analytics – Specialty, AWS Certified Developer)

Must have skills:

• Proficiency in AWS cloud services: AWS Glue, QuickSight, S3, Lambda, Athena, Redshift, EMR, and related technologies
• Strong experience with PySpark
• Expertise in SQL and data modeling for relational and non-relational databases
• Familiarity with business intelligence and visualization tools, especially Amazon QuickSight

Good to have:

• Proficiency in Python and ML libraries (scikit-learn, TensorFlow, PyTorch).
• Understanding of MLOps and model deployment best practices.
• Hands-on experience with AWS services for ML.
• Experience or familiarity with HVAC domain is a plus

Key Responsibilities:

• Design, develop, and maintain data pipelines using AWS Glue, PySpark, and related AWS services to extract, transform, and load (ETL) data from diverse sources
• Build and optimize data warehouse/data lake infrastructure on AWS, ensuring efficient data storage, processing, and retrieval
• Develop and manage ETL processes to source data from various systems, including databases, APIs, and file storage, and create unified data models for analytics and reporting
• Implement and maintain business intelligence dashboards using Amazon QuickSight, enabling stakeholders to derive actionable insights
• Collaborate with cross-functional teams (business analysts, data scientists, product managers) to understand requirements and deliver scalable data solutions
• Ensure data quality, integrity, and security throughout the data lifecycle, implementing best practices for governance and compliance5.
• Support self-service analytics by empowering internal users to access and analyze data through QuickSight and other reporting tools1.
• Troubleshoot and resolve data pipeline issues, optimizing performance and reliability as needed

Required Skills:

• Proficiency in AWS cloud services: AWS Glue, QuickSight, S3, Lambda, Athena, Redshift, EMR, and related technologies
• Strong experience with PySpark for large-scale data processing and transformation
• Expertise in SQL and data modeling for relational and non-relational databases
• Experience building and optimizing ETL pipelines and data integration workflows
• Familiarity with business intelligence and visualization tools, especially Amazon QuickSight
• Knowledge of data governance, security, and compliance best practices.
• Strong programming skills in Python; experience with automation and scripting
• Ability to work collaboratively in agile environments and manage multiple priorities effectively
• Excellent problem-solving and communication skills.

*******************************************************************************************************************************************