HDFS Command
Create a directory in HDFS
• mkdir
• Usage:
– hadoop fs -mkdir <paths>
• Example:
– hadoop fs -mkdir /user/saurzcode/dir1
/user/saurzcode/dir2
List the contents of a directory
• ls
• Usage :
– hadoop fs -ls <args>
• Example:
– hadoop fs -ls /user/saurzcode
Upload a file in HDFS
• put
• Usage:
– hadoop fs -put <localsrc> ... <HDFS_dest_Path>
• Example:
– hadoop fs -put /home/saurzcode/[Link]
/user/saurzcode/dir3/
Download a file in HDFS
• get
• Usage:
– hadoop fs -get <hdfs_src> <localdst>
• Example:
– hadoop fs -get /user/saurzcode/dir3/
[Link] /home/
See contents of a file
• cat
• Usage:
– hadoop fs -cat <path[filename]>
• Example:
– hadoop fs -cat /user/saurzcode/dir1/[Link]
Copy a file from source to destination
• cp
• Usage:
– hadoop fs -cp <source> <dest>
• Example:
– hadoop fs -cp /user/saurzcode/dir1/[Link]
/user/saurzcode/dir2
Copy a file from/To Local file system
to HDFS
• copyfromlocal
• Usage:
– hadoop fs -copyFromLocal <localsrc> URI
• Example:
– hadoop fs -copyFromLocal /home/saurzcode/
[Link] /user/saurzcode/[Link]
Move file from source to destination
• mv
• Usage :
– hadoop fs -mv <src> <dest>
• Example:
– hadoop fs -mv /user/saurzcode/dir1/[Link]
/user/saurzcode/dir2
Remove a file or directory in HDFS
• rm
• Usage :
– hadoop fs -rm <arg>
• Example:
– hadoop fs -rm /user/saurzcode/dir1/[Link]
Display the aggregate length of a file.
• du
• Usage :
– hadoop fs -du <path>
• Example:
– hadoop fs -du /user/saurzcode/dir1/[Link]
Change Replication Factor
• Setrep
• Usage:
• hadoop fs -setrep [-R] [-w] <numReplicas>
<path>
• Example:
• hadoop fs -setrep -w 3 /user/hadoop/dir1
Searching a file
• find
• Usage:
– hadoop fs -find <path> ... <expression> ...
• Example:
– hadoop fs -find / -name test –print
Displays free space
• df
• Usage:
– hadoop fs -df [-h] URI [URI ...]
• Example:
– hadoop dfs -df /user/hadoop/dir1
Count no. of directories, files and
bytes
• count
• Usage:
– hadoop fs -count [-q] [-h] [-v] <paths>
• -count –q options show:
– QUOTA, REMAINING_QUATA, SPACE_QUOTA, DIR_COUNT,
PATHNAME REMAINING_SPACE_QUOTA, FILE_COUNT,
CONTENT_SIZE
• The -h option shows sizes in human readable format.
• The -v option displays a header line.
• Example:
– hadoop fs -count -q hdfs://[Link]/file1
– hadoop fs -count -q -h hdfs://[Link]/file1
– hdfs dfs -count -q -h -v hdfs://[Link]/file1