The NameNode is the centerpiece of an HDFS file system. It keeps the directory tree of all files in the file system, and tracks where across the cluster the file data is kept. It does not store the data of these files itself.
Prerequisite
For Virtual Machines, install the Linux Agent.
Configuring the credentials
Configure the credentials in the directory /opt/opsramp/agent/conf/app.d/creds.yaml
hdfs-namenode:
- name: hdfs-namenode
user: <username>
pwd: <Password>
encoding-type: plain
labels:
key1: val1
key2: val2
Configure the application
Virtual machine
Configure the application in the directory /opt/opsramp/agent/conf/app/discovery/auto-detection.yaml
- name: hdfs-namenode
instance-checks:
process-check:
- namenode
port-check:
- 50070
- 9870
uri-check:
- "jmx?qry=Hadoop:service=NameNode,name=FSNamesystem*"
- "jmx?qry=Hadoop:service=NameNode,name=NameNodeInfo"
Docker environment
Configure the application in the directory /opt/opsramp/agent/conf/app/discovery/auto-container-detection.yaml
- name: hdfs-namenode
container-checks:
image-check:
- namenode
port-check:
- 50070
- 9870
Kubernetes environment
Configure the application in config.yaml
- name: hdfs-namenode
container-checks:
image-check:
- namenode
port-check:
- 50070
- 9870
Validate
Go to Resources under the Infrastructure tab to check if your resources are onboarded and the metrics are collected.
Supported metrics
OpsRamp Metric | Metric Display Name | Unit |
---|---|---|
hdfs_namenode_capacity_total Total disk capacity | Capacity Total | bytes |
hdfs_namenode_capacity_used Disk usage | Capacity Used | bytes |
hdfs_namenode_capacity_used_percent Disk usage | Capacity Used Percent | percent |
hdfs_namenode_capacity_remaining Remaining disk space left | Capacity Remaining | bytes |
hdfs_namenode_capacity_remaining_percent Remaining disk space left | Capacity Remaining Percent | percent |
hdfs_namenode_total_load Total load on the file system | Total Load | |
hdfs_namenode_fs_lock_queue_length Lock queue length | Fs Lock Queue Length | |
hdfs_namenode_blocks_total Total number of blocks | Blocks Total | Blocks |
hdfs_namenode_max_objects Maximum number of files that HDFS supports | Max Objects | |
hdfs_namenode_files_total Total number of files | Files Total | Files |
hdfs_namenode_pending_replication_blocks Number of blocks pending replication | Pending Replication Blocks | Blocks |
hdfs_namenode_under_replicated_blocks Number of under replicated blocks | Under Replicated Blocks | Blocks |
hdfs_namenode_scheduled_replication_blocks Number of blocks scheduled for replication | Scheduled Replication Blocks | Blocks |
hdfs_namenode_pending_deletion_blocks Number of pending deletion blocks | Pending Deletion Blocks | Blocks |
hdfs_namenode_num_live_data_nodes Total number of live data nodes | Num Live Data Nodes | Nodes |
hdfs_namenode_num_dead_data_nodes Total number of dead data nodes | Num Dead Data Nodes | Nodes |
hdfs_namenode_num_decom_live_data_nodes Number of decommissioning live data nodes | Num Decom Live Data Nodes | Nodes |
hdfs_namenode_num_decom_dead_data_nodes Number of decommissioning dead data nodes | Num Decom Dead Data Nodes | Nodes |
hdfs_namenode_volume_failures_total Total volume failures | Volume Failures Total | |
hdfs_namenode_estimated_capacity_lost_total Estimated capacity lost | Estimated Capacity Lost Total | bytes |
hdfs_namenode_num_decommissioning_data_nodes Number of decommissioning data nodes | Num Decommissioning Data Nodes | Nodes |
hdfs_namenode_num_stale_data_nodes Number of stale data nodes | Num Stale Data Nodes | Nodes |
hdfs_namenode_num_stale_storages Number of stale storages | Num Stale Storages | |
hdfs_namenode_missing_blocks Number of missing blocks | Missing Blocks | Blocks |
hdfs_namenode_corrupt_blocks Number of corrupt blocks | Corrupt Blocks | Blocks |
hdfs_namenode_num_failed_data_nodes Total number of failed data nodes | Num Failed Data Nodes | |
hdfs_namenode_nondfs_used_percent Total space used by NonDfs | Nondfs Used Percent | Percent |