Hadoop¶
| Hadoop Status Methods | URL |
|---|---|
| Get the current status of the Hadoop cluster. | GET /{cluster_id}/hadoop/status |
| Get a list of submitted jobs and their status. | GET /{cluster_id}/hadoop/jobs |
| Get the proxy port for the Hadoop UI | GET /{cluster_id}/hadoop/jt/port |
Hadoop Status¶
- GET /{cluster_id}/hadoop/status¶
Retrieve information about the status of the Hadoop cluster. Returns a JSON dictionary of cluster attributes and their current value.
Example:
curl http://127.0.0.1:8888/Test_Cluster/hadoop/status
Output:
{ "activeTrackerNames": [ "tracker_dse1:dse1/127.0.0.1:47333" ], "blacklisted_tracker_names": [], "has_recovered": false, "has_restarted": false, "hostname": "dse1", "http_port": 50030, "identifier": "201301301843", "mapTasks": 0, "max_map_tasks": 2, "max_memory": 4125097984, "max_reduce_tasks": 2, "num_active_trackers": 1, "num_blacklisted_trackers": 0, "num_excluded_nodes": 0, "reduce_tasks": 0, "start_time": 1359592991647, "state": 1, "task_tracker_expiry_interval": 600000, "total_submissions": 5, "used_memory": 4125097984 }
Job Details¶
- GET /{cluster_id}/hadoop/jobs¶
Returns a JSON list of submitted jobs, each of which is a dictionary of job attributes.
Example:
curl http://127.0.0.1:8888/Test_Cluster/hadoop/jobs
Output:
[ { "desired_maps": 1, "desired_reduces": 1, "duration": 18951, "finish_time": 0, "finished_maps": 1, "finished_reduces": 0, "job_id": { "as_string": "job_201301301843_0001", "job_id": 1, "job_tracker_id": "201301301843" }, "launch_time": 1359593329324, "priority": 2, "profile": { "job_file": "cfs:/tmp/hadoop-joe/mapred/staging/joe/.staging/job_201301301843_0001/job.xml", "name": "INSERT OVERWRITE TABLE 10da...b.column_name)(Stage-1)", "queue_name": "default", "user": "joe" }, "start_time": 1359593329128, "status": { "cleanup_progress": 0.0, "map_progress": 1.0, "reduce_progress": 0.0, "run_state": 1, "scheduling_info": "NA", "setup_progress": 1.0 } } ]