Find out actual disk usage in HDFS

Question

Is there a way to find out how much space is consumed in HDFS? I used

hdfs dfs -df

but it seems to be not relevant cause after deleting huge amount of data with

hdfs dfs -rm -r -skipTrash

the previous comand displays changes not at once but after several minutes (I need up-to-date disk usage info).

Solution 1

hadoop cli is deprecated. Use hdfs instead.

Folder wise :

sudo -u hdfs hdfs dfs -du -h /

Cluster wise :

sudo -u hdfs hdfs dfsadmin -report

Solution 2

To see the space consumed by a particular folder try:

hadoop fs -du -s /folder/path

And if you want to see the usage, space consumed, space available, etc. of the whole HDFS:

hadoop dfsadmin -report

Comments

[Aliaxander]

Unfortunately hadoop dfsadmin -report shows not up-to-date info too.

[Peter Krauss]

... And the correct command today (2019) is hdfs dfsadmin -report