Hadoop
| Hadoop Status Methods | URL |
|---|---|
Get the current status of the Hadoop cluster |
|
Get a list of submitted jobs and their status |
|
Get the proxy port for the Hadoop UI |
Hadoop Status
GET /{cluster_id}/hadoop/status
Retrieve information about the status of the Hadoop cluster. Returns a JSON dictionary of cluster attributes and their current value.
Example:
curl http://127.0.0.1:8888/Test_Cluster/hadoop/statusOutput:
{
"activeTrackerNames": [
"tracker_dse1:dse1/127.0.0.1:47333"
],
"blacklisted_tracker_names": [],
"has_recovered": false,
"has_restarted": false,
"hostname": "dse1",
"http_port": 50030,
"identifier": "201301301843",
"mapTasks": 0,
"max_map_tasks": 2,
"max_memory": 4125097984,
"max_reduce_tasks": 2,
"num_active_trackers": 1,
"num_blacklisted_trackers": 0,
"num_excluded_nodes": 0,
"reduce_tasks": 0,
"start_time": 1359592991647,
"state": 1,
"task_tracker_expiry_interval": 600000,
"total_submissions": 5,
"used_memory": 4125097984
}Job Details
GET /{cluster_id}/hadoop/jobs
Returns a JSON list of submitted jobs, each of which is a dictionary of job attributes.
Example:
curl http://127.0.0.1:8888/Test_Cluster/hadoop/jobsOutput:
[
{
"desired_maps": 1,
"desired_reduces": 1,
"duration": 18951,
"finish_time": 0,
"finished_maps": 1,
"finished_reduces": 0,
"job_id": {
"as_string": "job_201301301843_0001",
"job_id": 1,
"job_tracker_id": "201301301843"
},
"launch_time": 1359593329324,
"priority": 2,
"profile": {
"job_file": "cfs:/tmp/hadoop-joe/mapred/staging/joe/.staging/job_201301301843_0001/job.xml",
"name": "INSERT OVERWRITE TABLE 10da...b.column_name)(Stage-1)",
"queue_name": "default",
"user": "joe"
},
"start_time": 1359593329128,
"status": {
"cleanup_progress": 0.0,
"map_progress": 1.0,
"reduce_progress": 0.0,
"run_state": 1,
"scheduling_info": "NA",
"setup_progress": 1.0
}
}
]Proxy Port
GET /{cluster_id}/hadoop/jt/port
Retrieve the proxy port used by OpsCenter to expose the Hadoop HTTP UI.
Example:
curl http://127.0.0.1:8888/Test_Cluster/hadoop/jt/portOutput:
50031