Hadoop

Hadoop Status Methods	URL
Get the current status of the Hadoop cluster	GET /{cluster_id}/hadoop/status
Get a list of submitted jobs and their status	GET /{cluster_id}/hadoop/jobs
Get the proxy port for the Hadoop UI	GET /{cluster_id}/hadoop/jt/port

Hadoop Status Methods

URL

Get the current status of the Hadoop cluster

GET /{cluster_id}/hadoop/status

Get a list of submitted jobs and their status

GET /{cluster_id}/hadoop/jobs

Get the proxy port for the Hadoop UI

GET /{cluster_id}/hadoop/jt/port

Hadoop Status

GET /{cluster_id}/hadoop/status

Retrieve information about the status of the Hadoop cluster. Returns a JSON dictionary of cluster attributes and their current value.

Example:

 curl http://127.0.0.1:8888/Test_Cluster/hadoop/status

Output:

        {
            "activeTrackerNames": [
                "tracker_dse1:dse1/127.0.0.1:47333"
            ],
            "blacklisted_tracker_names": [],
            "has_recovered": false,
            "has_restarted": false,
            "hostname": "dse1",
            "http_port": 50030,
            "identifier": "201301301843",
            "mapTasks": 0,
            "max_map_tasks": 2,
            "max_memory": 4125097984,
            "max_reduce_tasks": 2,
            "num_active_trackers": 1,
            "num_blacklisted_trackers": 0,
            "num_excluded_nodes": 0,
            "reduce_tasks": 0,
            "start_time": 1359592991647,
            "state": 1,
            "task_tracker_expiry_interval": 600000,
            "total_submissions": 5,
            "used_memory": 4125097984
        }

Job Details

GET /{cluster_id}/hadoop/jobs

Returns a JSON list of submitted jobs, each of which is a dictionary of job attributes.

Example:

 curl http://127.0.0.1:8888/Test_Cluster/hadoop/jobs

Output:

        [
            {
                "desired_maps": 1,
                "desired_reduces": 1,
                "duration": 18951,
                "finish_time": 0,
                "finished_maps": 1,
                "finished_reduces": 0,
                "job_id": {
                    "as_string": "job_201301301843_0001",
                    "job_id": 1,
                    "job_tracker_id": "201301301843"
                },
                "launch_time": 1359593329324,
                "priority": 2,
                "profile": {
                    "job_file": "cfs:/tmp/hadoop-joe/mapred/staging/joe/.staging/job_201301301843_0001/job.xml",
                    "name": "INSERT OVERWRITE TABLE 10da...b.column_name)(Stage-1)",
                    "queue_name": "default",
                    "user": "joe"
                },
                "start_time": 1359593329128,
                "status": {
                    "cleanup_progress": 0.0,
                    "map_progress": 1.0,
                    "reduce_progress": 0.0,
                    "run_state": 1,
                    "scheduling_info": "NA",
                    "setup_progress": 1.0
                }
            }
        ]

Proxy Port

GET /{cluster_id}/hadoop/jt/port

Retrieve the proxy port used by OpsCenter to expose the Hadoop HTTP UI.

Example:

 curl http://127.0.0.1:8888/Test_Cluster/hadoop/jt/port

Output:

Hadoop

Hadoop Status

GET /{cluster_id}/hadoop/status

Job Details

GET /{cluster_id}/hadoop/jobs

Proxy Port

GET /{cluster_id}/hadoop/jt/port

Was this helpful?

Give Feedback