Associate Data Engineer
Job Description
•Degree in Computer Science or Computer Engineering • Minimum 5 years workingexperience in system operations compliance and management areas • Project hands-on experiencespecifically with AWS platform (primary requirement) • project experience in cloudoperations or cloud architecture • Must be cloud certified (AWS) CoreTechnical Skills: • proficiency in Databricksplatform, including workspace management, cluster configuration, and joborchestration • Strong expertise in ApacheSpark within Databricks environment, including Spark SQL, DataFrames, and RDDs • Good in-depth understanding ofdata warehouse concepts, data profiling, data verification and advancedanalytics techniques • Strong knowledge ofmonitoring, incident management, and cloud cost control TechnologyStack Experience: • Databricks • AWS cloud services andarchitecture • IDMC (Informatica DataManagement Cloud) • Tableau for data visualization • Oracle Database management • ML Ops practices withinDatabricks environment • STATA for statistical analysisis advantage • Amazon SageMaker integrationwith Databricks • DataRobot platformintegration
Responsibilities: Operational Responsibilities: • Monitor and maintainproduction data pipelines to ensure 99.9% uptime and optimal performance • Implement comprehensivelogging, alerting, and monitoring systems using Application monitoring tools • Perform regular health checksperformance, job execution times, and resource utilization to identify andresolve bottlenecks proactively • Manage incident responseprocedures for pipeline failures, including root cause analysis, resolution,and post-incident reviews • Establish and maintaindisaster recovery procedures and backup strategies for critical data assetswithin the Databricks environment
Thanks,and Best Regards KaranamVijaya Kiran (EARegistration no: R1443178) HP:+65 92333815 RecruitmentManager HeliusTechnologies Pte Ltd (EA Licence No: 11C3373) https://www.linkedin.com/in/vijay-karanam-68462131/