Consulting/Principal Software Engineer
Locations: Bengaluru
Time Type: Full time
Posted on: Posted 3 Days Ago
About our Team
Our Academic & Government team helps our communities accelerate knowledge for a better world by helping to establish, discover and advance knowledge. We do this by turning scientific discoveries into peer-reviewed knowledge, hosting the world’s richest corpus of knowledge, allowing users to unlock insights from content and by measuring the quality and impact of research - turning insights into outcomes.
BASIC FUNCTIONS:
This position provides high-level expertise and leadership in big data technologies and software engineering. The role involves developing and leading large multifunctional development activities, solving complex technical problems, architecting and implementing big data solutions, and serving as a senior source of expertise in big data technologies and cloud computing. The position may also provide sizing or budget recommendations to management.
QUALIFICATIONS:
- 10+ years of Software Engineering experience with a focus on big data technologies
- BS in Computer Science, Engineering, or equivalent experience required; advanced degree preferred
- Extensive experience with big data technologies, cloud computing, and distributed systems
TECHNICAL SKILLS:
- Expert in big data technologies, including Apache Spark, Hadoop ecosystem, and related frameworks
- Proficient in Java API development, with a focus on building scalable and performant applications
- Strong experience with Elasticsearch, including implementation, optimization, and management of large-scale search solutions
- Expertise in code optimization techniques for handling and processing large quantities of data
- Advanced knowledge of AWS and its services, particularly those related to big data and analytics (e.g., EMR, Redshift, Athena)
- Experience with Snowflake data warehouse (big plus)
- Proven expertise in troubleshooting and optimizing ETL (Extract, Transform, Load) pipeline issues
- Experience with graph databases, particularly Neo4j, including data modeling and query optimization
- Software development process expert in applicable methodologies (e.g., Agile, Waterfall)
- Expertise in data modeling, design and manipulation, optimization, best practices, and tuning for big data scenarios
- Strong knowledge of distributed systems architecture and design principles
- Proficiency in test-driven development and maintenance, including techniques for applying best practices in big data projects
- Expert knowledge of multiple data storage subsystems, including NoSQL databases and data warehouses
- Proven expertise in partnering and leading internal and external technology resources in solving complex big data challenges
- Strong experience with various resource models such as Managed Services and/or Staff Augmentation in cloud environments
- Solid knowledge of architectural principles for scalable and distributed systems
ACCOUNTABILITIES:
- Serve as a primary technical point of contact for big data initiatives and external technology resources
- Provide architectural and design input across big data products and solutions, including graph-based architectures
- Work directly with customers and end users to understand and translate business requirements into technical solutions
- Serve as the acknowledged "go-to" person on coding, big data technologies, and technical issues
- Interface with other technical personnel or team members to finalize requirements for big data projects
- Write and review detailed specifications for the development of complex big data system components
- Design, implement, and optimize big data processing pipelines using technologies like Spark, AWS services, Snowflake, and Neo4j
- Mentor lead software developers on big data development methodologies and optimization techniques
- Evaluate and recommend new technologies and approaches for handling large-scale data processing challenges
- Ensure best practices in data security, compliance, and governance are followed in all big data initiatives
- Collaborate with cross-functional teams to integrate big data solutions with existing systems and processes
- Troubleshoot and resolve complex issues in ETL pipelines, ensuring data integrity and performance
- Develop and implement graph-based data models and algorithms for complex relationship-based data scenarios
- All other duties as assigned
ADDITIONAL SKILLS:
- Strong proficiency with data manipulation languages and query optimization techniques for big data (e.g., Spark SQL, Hive, Snowflake SQL, Cypher for Neo4j)
- Experience with stream processing technologies (e.g., Kafka, Flink)
- Familiarity with machine learning libraries and their application in big data scenarios
- Knowledge of data visualization tools and techniques for large datasets, including graph visualization
- Understanding of data lake architectures and implementation strategies
- Expertise in designing and maintaining robust, scalable ETL processes across various big data platforms
- Ability to design and implement graph algorithms for data analysis and pattern recognition
The ideal candidate will have a proven track record of successfully designing and implementing big data solutions, preferably in a cloud environment, and will be able to lead teams in tackling complex data engineering challenges. Strong problem-solving skills and experience in optimizing data pipelines are essential. Expertise in both traditional big data systems and graph databases is highly valued for this role.
Work in a way that works for you
We promote a healthy work/life balance across the organisation. We offer an appealing working prospect for our people. With numerous wellbeing initiatives, shared parental leave, study assistance and sabbaticals, we will help you meet your immediate responsibilities and your long-term goals.
- Working flexible hours - flexing the times when you work in the day to help you fit everything in and work when you are the most productive
Working for you
We know that your wellbeing and happiness are key to a long and successful career. These are some of the benefits we are delighted to offer:
- Comprehensive Health Insurance: Covers you, your immediate family, and parents.
- Enhanced Health Insurance Options: Competitive rates negotiated by the company.
- Group Life Insurance: Ensuring financial security for your loved ones.
- Group Accident Insurance: Extra protection for accidental death and permanent disablement.
- Flexible Working Arrangement: Achieve a harmonious work-life balance.
- Employee Assistance Program: Access support for personal and work-related challenges.
- Medical Screening: Your well-being is a top priority.
- Modern Family Benefits: Maternity, paternity, and adoption support.
- Long-Service Awards: Recognizing dedication and commitment.
- New Baby Gift: Celebrating the joy of parenthood.
- Subsidized Meals in Chennai: Enjoy delicious meals at discounted rates.
- Various Paid Time Off: Take time off with Casual Leave, Sick Leave, Privilege Leave, Compassionate Leave, Special Sick Leave, and Gazetted Public Holidays.
- Free Transport pick up and drop from the home -office - home (applies in Chennai)
About the Business
A global leader in information and analytics, we help researchers and healthcare professionals advance science and improve health outcomes for the benefit of society. Building on our publishing heritage, we combine quality information and vast data sets with analytics to support visionary science and research, health education and interactive learning, as well as exceptional healthcare and clinical practice. At Elsevier, your work contributes to the world’s grand challenges and a more sustainable future. We harness innovative technologies to support science and healthcare to partner for a better world.