Data Software Engineer - Spark, Scala

DESCRIPTION

Join our team at EPAM!

Where you have the unique opportunity to work alongside world-class experts and technologies right from the heart of Silicon Valley.

We are seeking a Senior/Lead Software Data Engineer focused on software engineering rather than data science or analytics.

In this role, you will be pivotal in writing software, developing data processing jobs, and constructing data pipelines for both internal and external partners.

We prioritize the quality of the data produced as well as the cleanliness and manageability of the code, recognizing data as our primary product.

EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential.

Want more jobs like this?

Get Science and Engineering jobs in Río Grande, Mexico delivered to your inbox every week.

Get Jobs

Send me The Muse newsletters for the best in career advice and job search tips.

By signing up, you agree to our Terms of Service & Privacy Policy.

• Fix, change, or add configuration behaviors, such as Spark properties or job quality metrics
• Maintain Spark job functionalities by addressing bugs, modifying upstream datasets, and updating data schemas as necessary
• Implement minor changes in the source code using various Spark APIs for Scala
• Collaborate with cross-functional teams to ensure the accuracy and quality of data products
• Participate in code reviews and contribute to maintaining high standards of code quality

• Advanced proficiency in Scala (version 2.12)
• Intermediate proficiency in Apache Spark (versions 3.2 and 3.4), with a solid understanding of Datasets, DataFrames, and RDDs
• Basic to intermediate proficiency in HDFS, with knowledge of data storage and access through system accounts
• Basic understanding of Apache Airflow, capable of adjusting scheduled actions in data pipelines
• Basic proficiency in Git for version control tasks, such as clone, pull, add, commit, push, and rebase
• Basic proficiency in Jenkins for understanding the workflow of binary generation and feature deployment

• Basic understanding of Hive Metastore (HMS) tables
• Experience with Github or other source version control technologies
• Experience with Github or other source version control technologies

• Scala (version 2.12)
• Apache Spark (versions 3.2 and 3.4)
• HDFS
• Apache Airflow
• Git
• Jenkins

• Career plan and real growth opportunities
• Unlimited access to LinkedIn learning solutions
• International Mobility Plan within 25 countries
• Constant training, mentoring, online corporate courses, eLearning and more
• English classes with a certified teacher
• Support for employee's initiatives (Algorithms club, toastmasters, agile club and more)
• Enjoyable working environment (Gaming room, napping area, amenities, events, sport teams and more)
• Flexible work schedule and dress code
• Collaborate in a multicultural environment and share best practices from around the globe
• Hired directly by EPAM & 100% under payroll
• Law benefits (IMSS, INFONAVIT, 25% vacation bonus)
• Major medical expenses insurance: Life, Major medical expenses with dental & visual coverage (for the employee and direct family members)
• 13 % employee savings fund, capped to the law limit
• Grocery coupons
• 30 days December bonus
• Employee Stock Purchase Plan
• 12 vacations days plus 4 floating days
• Official Mexican holidays, plus 5 extra holidays (Maundry Thursday and Friday, November 2nd, December 24th & 31st)
• Relocation bonus: transportation, 2 weeks of accommodation for you and your family and more
• Monthly non-taxable amount for the electricity and internet bills

• By applying to our role, you are agreeing that your personal data may be used as in set out in EPAM's Privacy Notice and Policy