Important IT company At the Latin American level, growth requires:

Spark EKS Engineer

Job description:

We are seeking a Spark lead focused on operations to administer/scale our multi-petabyte Spark EKS clusters and related services that go with it. This role focuses primarily on provisioning, ongoing capacity planning, monitoring, management of Spark EKS platform running on AWS and performance enhancement of application/middleware that runs on this platform.

Key Qualifications:

• Well versed with AWS - EMR/ S3, and other AWS services and dashboards, At least AWS administrator level
• Preferred - AWS certification for EMR/ EKS cluster management
• Responsible for maintaining large scale (1000+ nodes) production Spark clusters
• Point of contact for all Spark related issues coming from Application teams and internal clusters, responsible for troubleshooting and recommendation for Spark and MR jobs. Should be able to use existing logs to debug the issue.
• Responsible for implementation and ongoing administration of Spark, Flink & Trino infrastructure including monitoring, tuning and troubleshooting
• Improve scalability, service reliability, capacity, and performance of the cluster and applications running in the cluster
• Triage production issues when they occur with other operation and engineering teams.
• Conduct ongoing maintenance across our large scale deployments across the world
• Write automation code for managing large Big Data clusters
• Participate in on-call rotation
• Hands on experience to troubleshoot incidents, formulate theories and test hypothesis, and narrow down possibilities to find the root cause.
• Deep understanding of Spark Eco system

• Hands on experience with managing production clusters (Hadoop, Spark).
• Strong development/automation skills. Must be very comfortable with reading and writing Python code/ Scripting.
• At least 5+ years of Spark experience in large scale, multi-tenant production clusters (1000+ instances)
• Tools-first mindset. You build tools for yourself and others to increase efficiency and to make hard or repetitive tasks easy and quick.
• Experience with Configuration Management and automation.
• Organized, focused on building, improving, resolving and delivering.

ADVANCED CONVERSATIONAL ENGLISH ESSENTIAL (Will be evaluated).

Job type: Mostly remote

Location: GDL- Monterrey- Mexico city (any of this Cities)

Salary: $100,000 gross.

Benefits: Excellent superior benefits.