Senior / Software Engineer (Spark Developer) Singapore, Singapore
Senior / Software Engineer (Spark Developer) Description
Job #: 57562DESCRIPTION
We are looking for an experienced Senior / Software Engineer (Spark Developer) in Singapore to make the team stronger.
Responsibilities
- Understand the Business requirements, Functional and Technical requirements and translate them effectively using the Spark Framework
- Understand the complex transformation logic and translate them to Hive/Spark-SQL queries
- Deal with live streaming datasets using Kafka/Spark Streaming
Requirements
- 2 to 9 years’ hands on experience on Spark Core, Spark-SQL, Scala-Programming and Streaming datasets in Big Data platform
- Should have extensive working experience in Hive and other components of Hadoop eco system (HBase, Zookeeper, Kafka and Flume)
- Should be able to understand the complex transformation logic and translate them to Spark-SQL queries
- Unix Shell Scripting and setting up CRON jobs
- Should have worked on Cloudera distribution framework, Oozie Workflow (or any Scheduler), Jenkins (or any version controller)
- Prior experience in Consumer Banking Domain is an advantage
- Prior experience in agile delivery method is an advantage
- Excellent understanding of technology life cycles and the concepts and practices required to build big data solutions
- Familiar with Data Warehouse concepts
- Core Java skillset is an added advantage
- Good Knowledge and experience in any database (Teradata or Data Lake or Oracle or SQL Server) is a plus
- Bachelor of Engineering, Bachelor of Technology, MCA (or equivalent master degree)
We offer
- Friendly team and enjoyable working environment
- Work-life balance and flexible schedule
- Online training library, mentoring, career development and potential partial grant of certification
- Unlimited access to LinkedIn learning solutions
- Referral bonuses
- Compensation for sick leave and paid time off
- Opportunities for self-realization