Hadoop Administrator

Job Title: Senior Software Engineer

Experience: 8 – 11 years

Location: SFO/ Houston

Primary Skills:

1. Set up production Hadoop clusters with optimum configurations
2. Drive automation of Hadoop deployments, cluster expansion and maintenance operations
3. Manage Hadoop cluster, monitoring alerts and notification
4. Deployment of upgrades, updates and patches
5. Provide 24x7 tier-3 troubleshooting and break-fix support for production services
6. Diagnosis of installation & configuration issues
7. Diagnosis of cluster management issues
8. Diagnosis of performance issues
9. Job scheduling, monitoring, debugging and troubleshooting
10. Monitoring and management of the cluster in all respects, notably availability, performance and security
11. File system management and monitoring
12. Data transfer between Hadoop and other data stores (incl. relational database)
13. Ingesting additional data sources into Hadoop in either streaming or batch mode
14. Commissioning and decommissioning worker nodes
15. Configuring the cluster to be rack-aware
16. Set up High Availability/Disaster Recovery environment
17. Debug/Troubleshoot environment failures/downtime
18. Working with data delivery teams to setup new Hadoop users, including setting up Linux users, Kerberos principals, and testing HDFS, Hive, Pig, and MapReduce access for new users
19. Set up security configurations in Hadoop
20. Performance tuning of Hadoop clusters and Hadoop Map Reduce routines
21. Manage and review Hadoop log files

Qualifications

Engineering Graduate, MCA, Masters/Post Graduates (preferably IT/ CS)